From patchwork Fri Sep 13 13:05:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 828329 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 724FB1DA62B; Fri, 13 Sep 2024 13:05:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232752; cv=none; b=E84oH0tOwFhjGbyRPbfksxSe3ef/kZQuj4xwrd/aQz9LTLJ1vqvHr/N/soNPHxnsUXDAu1Z7Ezj2Zm4fY9PuYAZmlXOomunu9mKefApXorC5czC6o+wh+WCChahd9WPRp7R9tu+Y7tcV5/uNcxhq5zStvc79cgDrEgFePsCsfNY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232752; c=relaxed/simple; bh=ewIwlHsGLmz+83HdU8wNpYP0fBOT/s5U2JGtYZzcZ90=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O0mtu84bCbadzIBG9qcQGyY6bRKVwaHsESHjX5SzPMOMRct8O1NUBpu8RcoJSA/F67rNG/jC2tp4I4PVHvPVSlZu4qYJj7tDBnZSOiWlNkHOPnlyMYJkFzxrMpPuO14SMG41Ytd8ObEvVjFKns8/MTaK71d4C9gsymHLtylEJ08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=h22x0Xwa; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="h22x0Xwa" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48DBIxfx006405; Fri, 13 Sep 2024 13:05:49 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=5R8W6hZbVAmew KZX76CK+Ydz/RkbMlCfYBPDoaMlRD0=; b=h22x0XwaEgW5Pzv2CST2DHqV0tNTK MgixS3YlojOdCvEzhCAXUVCMOzHyEXMFdrISyUB2xznsFDKKWRSEUUrlwov4aQ6E ES7F9Qvixdpa0G9KFnMTjpQV7WpkRoU6KTcpjH9en3vIBDQjpyh+WYPGWBUB0dpn uLtwhfQqKYHd6pI6epB052KKNdIY1TreuiKvgQrLfw00wO6d8cj71XihcYe7xZPD dccdanMnsQBE6VNLpSQgQNvf6tELtwsQ39KUJMFxZ4sAnNDHMaSjKMVUTkXlS3+c ITyXkzvt1ZFf5/3sDZxSBN7sL14/VHA8X0lLymfCLFvsktfXamAkposiw== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41geg01ugf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:48 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48DCpAs7013465; Fri, 13 Sep 2024 13:05:48 GMT Received: from smtprelay03.fra02v.mail.ibm.com ([9.218.2.224]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 41h3cmp1fy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:48 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay03.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5iDs43778536 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:45 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D9E8120040; Fri, 13 Sep 2024 13:05:44 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 896AE2004B; Fri, 13 Sep 2024 13:05:44 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:44 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 1/7] s390/facility: Disable compile time optimization for decompressor code Date: Fri, 13 Sep 2024 15:05:37 +0200 Message-ID: <20240913130544.2398678-2-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: hYx6rSQVGViTaEbWbHnYUK-J20iR-Mhx X-Proofpoint-ORIG-GUID: hYx6rSQVGViTaEbWbHnYUK-J20iR-Mhx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_09,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 lowpriorityscore=0 suspectscore=0 mlxlogscore=570 priorityscore=1501 adultscore=0 clxscore=1011 spamscore=0 bulkscore=0 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130087 Disable compile time optimizations of test_facility() for the decompressor. The decompressor should not contain any optimized code depending on the architecture level set the kernel image is compiled for to avoid unexpected operation exceptions. Add a __DECOMPRESSOR check to test_facility() to enforce that facilities are always checked during runtime for the decompressor. Reviewed-by: Sven Schnelle Signed-off-by: Heiko Carstens --- arch/s390/include/asm/facility.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/arch/s390/include/asm/facility.h b/arch/s390/include/asm/facility.h index b7d234838a36..65ebf86506cd 100644 --- a/arch/s390/include/asm/facility.h +++ b/arch/s390/include/asm/facility.h @@ -59,8 +59,10 @@ static inline int test_facility(unsigned long nr) unsigned long facilities_als[] = { FACILITIES_ALS }; if (__builtin_constant_p(nr) && nr < sizeof(facilities_als) * 8) { - if (__test_facility(nr, &facilities_als)) - return 1; + if (__test_facility(nr, &facilities_als)) { + if (!__is_defined(__DECOMPRESSOR)) + return 1; + } } return __test_facility(nr, &stfle_fac_list); } From patchwork Fri Sep 13 13:05:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 829472 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EBBBC1DA62F; Fri, 13 Sep 2024 13:05:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232754; cv=none; b=h9PmflSNOx8fAFLEVIBndJK+E94DY2xRciPsD0hokg67lyAYFDL2Hy6d3bbmL+0hMGGknvPIg/NrXGoobdg8aCfX9gZfSPYWbk7GZMhyVJF4FCkroMu5jlkdzGBASk0G2iMHKHRFVgSHkmSQu+fDQT/sibsTiyKL5ITa5taTEJA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232754; c=relaxed/simple; bh=oEl36+voz+qtTGEMNvzWqOVzXnXDCugeAkPm19j4gFo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F2TFODYZNaQE0kmMQcJnqba7eggcqOrYifYpbuCzkdqqAvmiD8GCXxvaiWzepI+sY8Cox/8mXyM3SjUAzG2ATs2sU6Rc+fQQsq8a3WB5kRrL7ybG50UEg9JbnOT5AG7SaVAmRqXQnRiV1w4bynuSBmANFVT8M0hO5rup6JekzAU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=a92qGqf1; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="a92qGqf1" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48D6GP55019737; Fri, 13 Sep 2024 13:05:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=HuklU5+6a/WK1 HB92pTW2oH1NQRKsayz43JC79nFsRg=; b=a92qGqf1PubJc5b1e2nelnoWH2ZnU NIwIDwyo+B+PSUuqVaXT+mWkcsVCaqL/7hrxI/g347jW4+FMjF3dwdhbPXXVfWBT 8CL/2TV7yDBPrRoRhFm6z0pBNiWQ5Uo153ZrBL3Eu/mZfQ5GSYpay67WNmTNTsCh NWOr1LVGylnNjLOfaNewyKyk2GC2zVOVDoO80oXJlStsFe0vy1Z3vSWv8gVihNUT usc+Y1oe3f6glzgTM4G4VaD54HOJnsS3iZe7mbMXoEtUnOO+fMkERqacJ7QxxtNI ekRzsVJxRFZ09oingpIPqJgNslzHu3MecvDtIzaNXe9I/LAgTBYsyJODg== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41gebat3hh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:49 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48DB04Xc003123; Fri, 13 Sep 2024 13:05:48 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 41h15uedrm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:48 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5jcP31457842 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:45 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 36DA220040; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E435420043; Fri, 13 Sep 2024 13:05:44 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:44 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 2/7] s390/alternatives: Remove ALT_FACILITY_EARLY Date: Fri, 13 Sep 2024 15:05:38 +0200 Message-ID: <20240913130544.2398678-3-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: jv36OOvX-Fhkyxh3JKZDyR-8Gt1ihtNo X-Proofpoint-GUID: jv36OOvX-Fhkyxh3JKZDyR-8Gt1ihtNo X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_10,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 priorityscore=1501 bulkscore=0 spamscore=0 phishscore=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=715 impostorscore=0 clxscore=1015 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130091 Patch all alternatives which depend on facilities from the decompressor. There is no technical reason which enforces to split patching of such alternatives to the decompressor and the kernel. This simplifies alternative handling a bit, since one alternative type is removed. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/alternative.h | 6 +----- arch/s390/kernel/entry.S | 2 +- 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/arch/s390/include/asm/alternative.h b/arch/s390/include/asm/alternative.h index de980c938a3e..73e781b56bfe 100644 --- a/arch/s390/include/asm/alternative.h +++ b/arch/s390/include/asm/alternative.h @@ -39,11 +39,7 @@ #define ALT_TYPE_SHIFT 20 #define ALT_CTX_SHIFT 28 -#define ALT_FACILITY_EARLY(facility) (ALT_CTX_EARLY << ALT_CTX_SHIFT | \ - ALT_TYPE_FACILITY << ALT_TYPE_SHIFT | \ - (facility) << ALT_DATA_SHIFT) - -#define ALT_FACILITY(facility) (ALT_CTX_LATE << ALT_CTX_SHIFT | \ +#define ALT_FACILITY(facility) (ALT_CTX_EARLY << ALT_CTX_SHIFT | \ ALT_TYPE_FACILITY << ALT_TYPE_SHIFT | \ (facility) << ALT_DATA_SHIFT) diff --git a/arch/s390/kernel/entry.S b/arch/s390/kernel/entry.S index 749410cfdbc0..269436665d02 100644 --- a/arch/s390/kernel/entry.S +++ b/arch/s390/kernel/entry.S @@ -42,7 +42,7 @@ _LPP_OFFSET = __LC_LPP .macro LPSWEY address, lpswe ALTERNATIVE_2 "b \lpswe;nopr", \ - ".insn siy,0xeb0000000071,\address,0", ALT_FACILITY_EARLY(193), \ + ".insn siy,0xeb0000000071,\address,0", ALT_FACILITY(193), \ __stringify(.insn siy,0xeb0000000071,LOWCORE_ALT_ADDRESS+\address,0), \ ALT_LOWCORE .endm From patchwork Fri Sep 13 13:05:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 829474 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7DF7F1DA633; Fri, 13 Sep 2024 13:05:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232753; cv=none; b=VKz0K74/PvaWQeUmPFC24aCYZsk1V31fChG1TZz1/D+/1xpL+5dJiITlYOJdgUmF9LOPzoe2103WXgMrBTN5Au8eB+6PXoGu+xRu5JWJPuOiMwx8jzeLxkeM2ZfDZZH6ri5SecgHD4fLolm1mAC95WiMx6SNvV1n22why/MV/bk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232753; c=relaxed/simple; bh=ZeSBXGnKWmELJd9VKFcca2xHwEh3bOM2Y8tNSdtl5A0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UQOUT/ZbPyCtODR24p3B5sr4YM4kn6uPW9iGvxgaWeMObtQJdesvJE+o+HhZMeUdd3SDyph1UoL07uk+7hiPuLTL3dblw5UZ5qTbLj+k5RZnjN2jBU68r/HA/fFI22iPraKnPtaEYtmmj9JU0E0ArmFrFy09l9DJ6MgJpCizKNY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=U5voN1IV; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="U5voN1IV" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48D69d0v017485; Fri, 13 Sep 2024 13:05:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=8G8U5MZwRX6WH tIt0vxDGdxnDdLkEPMl63e5SJO3lbo=; b=U5voN1IV2nc5iybd99Bspnz8NMW1N xjXcWMA2e1Sf6zRLp4HnNWO3V9J+pGl904ysfJUUKCPxp5CRPg6Y7oZ/kC2lpY5C ZnfSxCaofDcUdHqkvWy9aH+/SiQWlcj5JQ8Bp3bbVD7bjXN3jnbxzbIHRVJwhJpt L5TALSK3/ykxBjgwiRw5we2fHYuIF08EBtOjYP+Tkch9go0gkTbFqlV/H5Qy2Y5k C9aFUJYhodEj9qMnCqSHGxtcAbiBf3APuwJjof+ojlhLGfgbxAsM42nnHPimKZg6 UZ6a8Os7Nx1AINthg3dYiE9SfuuCISZhywuLt0P3ZlNHtV4G85eAP2F4Q== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41gejb2521-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:50 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48DCFWEH010793; Fri, 13 Sep 2024 13:05:49 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 41kmb71vj2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:49 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5jOO10944802 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:45 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 87E4020040; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4166F2004B; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 3/7] s390/facility: Let test_facility() generate static branch if possible Date: Fri, 13 Sep 2024 15:05:39 +0200 Message-ID: <20240913130544.2398678-4-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: AwyVxn2EB-Vs-6MZ-XBh0ZU2WUv0a4uJ X-Proofpoint-GUID: AwyVxn2EB-Vs-6MZ-XBh0ZU2WUv0a4uJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_10,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 mlxscore=0 phishscore=0 mlxlogscore=737 spamscore=0 impostorscore=0 adultscore=0 bulkscore=0 suspectscore=0 malwarescore=0 lowpriorityscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130091 Let test_facility() generate a branch instruction if the tested facility is a constant, and where the result cannot be evaluated during compile time. The branch instruction defaults to "false" and is patched to nop (branch not taken) if the tested facility is available. This avoids runtime checks and is similar to x86's static_cpu_has() and arm64's alternative_has_cap_likely(). Signed-off-by: Heiko Carstens --- arch/s390/include/asm/facility.h | 37 +++++++++++++++++++++++++------- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/arch/s390/include/asm/facility.h b/arch/s390/include/asm/facility.h index 65ebf86506cd..715bcf8fb69a 100644 --- a/arch/s390/include/asm/facility.h +++ b/arch/s390/include/asm/facility.h @@ -14,7 +14,7 @@ #include #include #include - +#include #include #define MAX_FACILITY_BIT (sizeof(stfle_fac_list) * 8) @@ -39,30 +39,51 @@ static inline void __clear_facility(unsigned long nr, void *facilities) ptr[nr >> 3] &= ~(0x80 >> (nr & 7)); } -static inline int __test_facility(unsigned long nr, void *facilities) +static __always_inline bool __test_facility(unsigned long nr, void *facilities) { unsigned char *ptr; if (nr >= MAX_FACILITY_BIT) - return 0; + return false; ptr = (unsigned char *) facilities + (nr >> 3); return (*ptr & (0x80 >> (nr & 7))) != 0; } +/* + * __test_facility_constant() generates a single instruction branch. If the + * tested facility is available (likely) the branch is patched into a nop. + * + * Do not use this function unless you know what you are doing. All users are + * supposed to use test_facility() which will do the right thing. + */ +static __always_inline bool __test_facility_constant(unsigned long nr) +{ + asm goto( + ALTERNATIVE("brcl 15,%l[l_no]", "brcl 0,0", ALT_FACILITY(%[nr])) + : + : [nr] "i" (nr) + : + : l_no); + return true; +l_no: + return false; +} + /* * The test_facility function uses the bit ordering where the MSB is bit 0. * That makes it easier to query facility bits with the bit number as * documented in the Principles of Operation. */ -static inline int test_facility(unsigned long nr) +static __always_inline bool test_facility(unsigned long nr) { unsigned long facilities_als[] = { FACILITIES_ALS }; - if (__builtin_constant_p(nr) && nr < sizeof(facilities_als) * 8) { - if (__test_facility(nr, &facilities_als)) { - if (!__is_defined(__DECOMPRESSOR)) - return 1; + if (!__is_defined(__DECOMPRESSOR) && __builtin_constant_p(nr)) { + if (nr < sizeof(facilities_als) * 8) { + if (__test_facility(nr, &facilities_als)) + return true; } + return __test_facility_constant(nr); } return __test_facility(nr, &stfle_fac_list); } From patchwork Fri Sep 13 13:05:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 828326 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C8331DA631; Fri, 13 Sep 2024 13:05:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232754; cv=none; b=gJOvEQr2Mj2sUXvp4txnZ+FhOWOU0OxOqU6UxKB6pG95JL0fxwfGv/sV2HTjqYFhxu9Fn7DZrBn2HlabT9vawugKyHacxyb9ViNCYVb2OLPdGd07kPQI/KXch8BJZROueAdwEUkm1/BM83kocX65cwYpdhDyBxxxgahM5x50wgU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232754; c=relaxed/simple; bh=tk7+gVZLeLXU7t+66s8i13HI1nku9D8W8bICtBD60gg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cPzBTxxsVsorukNjzQhM1+HeGhWEz+sU/FvWgLJDeUNrPRobEcibkb3FicVacB2Ni1SvGp3uma5eckrsR+O0GxzsaBvxNrIK4ObyDHITKqUDCRaf2g+xsBLG3It1MpRjsU1k7y+9GzFSTinZrmTV3eu7jSf8p7hZ6PDUQ5YYyTg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=gixD29Vp; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="gixD29Vp" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48D8CDa9021760; Fri, 13 Sep 2024 13:05:50 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=uaXYr7IRkkEcw gpPxsCQAfW/yt2dgirIHku/Is3oCQ8=; b=gixD29Vpsn5GJygQRNZrOy4VuMQ88 KEn+gs5Wz4uZC59Tk58IiG2+8d+ToCGTRcne22dmOr6JHCqW6Mv/gG9pDRYCaJrR tqNruoOiExOPYrI5eGYGkT3T0SULYmgiRQrJ1dUBBv7PoKK4P8RV0KMyuNvjG65T +LBSYRzptTQKjtGN2XlW1d4KvG6N8O7DBuZvnCXXUzjtOVzAPIdAvElxqCXb69kL dJpsUPirdnpEzvCUfBfID1ujfBrnQZed+HFE2RYh/+F8h7eH9qGqI4q3dgI1E9qR YdSiJa1oRpyziw/X3IHmUlMDfaPs8PO2NvpVpKR+EN1g2YU0GmXNsltHA== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41gc8qtamk-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:50 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48DC5KWD032099; Fri, 13 Sep 2024 13:05:49 GMT Received: from smtprelay04.fra02v.mail.ibm.com ([9.218.2.228]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 41h2nn666j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:49 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay04.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5jDb10093024 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:46 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E145420040; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 923AB20043; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 4/7] s390/module: Provide find_section() helper Date: Fri, 13 Sep 2024 15:05:40 +0200 Message-ID: <20240913130544.2398678-5-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: qdAT7z5y4BsTCTJ4ZyG4GIw5S5GdtQPL X-Proofpoint-ORIG-GUID: qdAT7z5y4BsTCTJ4ZyG4GIw5S5GdtQPL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_09,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 spamscore=0 mlxscore=0 lowpriorityscore=0 impostorscore=0 priorityscore=1501 malwarescore=0 adultscore=0 clxscore=1015 mlxlogscore=606 suspectscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130087 Provide find_section() helper function which can be used to find a section by name, similar to other architectures. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/module.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/arch/s390/include/asm/module.h b/arch/s390/include/asm/module.h index 9f1eea15872c..916ab59e458a 100644 --- a/arch/s390/include/asm/module.h +++ b/arch/s390/include/asm/module.h @@ -38,4 +38,18 @@ struct mod_arch_specific { #endif /* CONFIG_FUNCTION_TRACER */ }; +static inline const Elf_Shdr *find_section(const Elf_Ehdr *hdr, + const Elf_Shdr *sechdrs, + const char *name) +{ + const char *secstrs = (void *)hdr + sechdrs[hdr->e_shstrndx].sh_offset; + const Elf_Shdr *s, *se; + + for (s = sechdrs, se = sechdrs + hdr->e_shnum; s < se; s++) { + if (strcmp(name, secstrs + s->sh_name) == 0) + return s; + } + return NULL; +} + #endif /* _ASM_S390_MODULE_H */ From patchwork Fri Sep 13 13:05:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 829473 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 382311DA63A; Fri, 13 Sep 2024 13:05:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232753; cv=none; b=Mbbijp2QT8otMpVh4cr+FazLddObgGBkAovRuQ3ZHYZiznfsiUNnTL+bUEjWX7Uw7lWK2Fvo+gokOy9cxyDSsU689+v65CdrFLyEcADPIImfTioIOLJVzlJtmQQNnyuqjR+bwnnmpT9dr3um0EBGLwxfeJBaucaJzyx7EpUoPT8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232753; c=relaxed/simple; bh=XuYhqwU7zJfFWBBPFkJwQ+AWb6A0cRgb3KCbwUj0fkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mvvspBKnVP7EzAgKXs0Lcg1ECNMq/j8p/HOg70iKeCABaJQBQYVQGyBnmnbPEsEdA8/TA9/TW6A1ZvqMDlgGTwkiJKI1i8MnM6NY6qR8H0//eFm9GTy7oDd8HoxeEuDru6D06tjaXH6bJ6wVE3Ba3+B711Vd+I7P6GnNY1Hp4HA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=K9l8/rwl; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="K9l8/rwl" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48D7nsSE013634; Fri, 13 Sep 2024 13:05:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=lLFW1eSbEh2sb EVG/GFtT5Z0LU4t4XDUY8zrqEjn5Lo=; b=K9l8/rwlF0nnL1TdaSHmiUGnUtHpi q0sJRAh3wDMtA6OJpoJguDIqL7ZYVINYqgU6f62kNFXZzAnAOVOVg6klDNo/HMSX QxmIyb0XZyq+vIDcBJd/3nOIUfhQPOKEeymunX7OQHiAKKUdfTPU2v3bQOFpXYuf Zz9fs8Leo9mdN4coSV8F16dVvCFasFjYykrwbfTndlWZJ9dKIZt0PHyZR77cDN0K BRJfT3pGRvS4YsyCCsS7G38iDz9g5D+Mmp/13L1XiQ3lo7vxTZxtGwKFWx+q5vKi 63QgZEGAAASQ5bgvBiy2rPkPuesxihSt5l0um17zZzlAhHsy86sB8ItJQ== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41gd8m1w4r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:50 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48DC1m5G032073; Fri, 13 Sep 2024 13:05:50 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 41h2nn666k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:50 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5kTR36372758 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:46 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3D2F120040; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EB9012004B; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:45 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 5/7] s390/vdso: Allow alternatives in vdso code Date: Fri, 13 Sep 2024 15:05:41 +0200 Message-ID: <20240913130544.2398678-6-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: GK6W9mLi6BybWCboA39Wm71uXlt-4Hxq X-Proofpoint-ORIG-GUID: GK6W9mLi6BybWCboA39Wm71uXlt-4Hxq X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_09,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 suspectscore=0 clxscore=1015 phishscore=0 spamscore=0 lowpriorityscore=0 adultscore=0 mlxscore=0 mlxlogscore=913 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130087 Implement the infrastructure required to allow alternatives in vdso code. Signed-off-by: Heiko Carstens --- arch/s390/kernel/vdso.c | 19 +++++++++++++++++++ arch/s390/kernel/vdso64/vdso64.lds.S | 4 ++++ 2 files changed, 23 insertions(+) diff --git a/arch/s390/kernel/vdso.c b/arch/s390/kernel/vdso.c index 2f967ac2b8e3..8e4e6b316337 100644 --- a/arch/s390/kernel/vdso.c +++ b/arch/s390/kernel/vdso.c @@ -12,12 +12,14 @@ #include #include #include +#include #include #include #include #include #include #include +#include #include extern char vdso64_start[], vdso64_end[]; @@ -250,8 +252,25 @@ static struct page ** __init vdso_setup_pages(void *start, void *end) return pagelist; } +static void vdso_apply_alternatives(void) +{ + const struct elf64_shdr *alt, *shdr; + struct alt_instr *start, *end; + const struct elf64_hdr *hdr; + + hdr = (struct elf64_hdr *)vdso64_start; + shdr = (void *)hdr + hdr->e_shoff; + alt = find_section(hdr, shdr, ".altinstructions"); + if (!alt) + return; + start = (void *)hdr + alt->sh_offset; + end = (void *)hdr + alt->sh_offset + alt->sh_size; + apply_alternatives(start, end); +} + static int __init vdso_init(void) { + vdso_apply_alternatives(); vdso64_mapping.pages = vdso_setup_pages(vdso64_start, vdso64_end); if (IS_ENABLED(CONFIG_COMPAT)) vdso32_mapping.pages = vdso_setup_pages(vdso32_start, vdso32_end); diff --git a/arch/s390/kernel/vdso64/vdso64.lds.S b/arch/s390/kernel/vdso64/vdso64.lds.S index 37e2a505e81d..fa02c6ae3cac 100644 --- a/arch/s390/kernel/vdso64/vdso64.lds.S +++ b/arch/s390/kernel/vdso64/vdso64.lds.S @@ -42,6 +42,10 @@ SECTIONS .rodata : { *(.rodata .rodata.* .gnu.linkonce.r.*) } .rodata1 : { *(.rodata1) } + . = ALIGN(8); + .altinstructions : { *(.altinstructions) } + .altinstr_replacement : { *(.altinstr_replacement) } + .dynamic : { *(.dynamic) } :text :dynamic .eh_frame_hdr : { *(.eh_frame_hdr) } :text :eh_frame_hdr From patchwork Fri Sep 13 13:05:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 828327 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB77C1DA635; Fri, 13 Sep 2024 13:05:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232754; cv=none; b=kzGQuR471Pqw/lQAJBf+tVyFXngfZ1U+dBqKu+lVCVB2Q6pjmJk/DfnSOnBhB1+BA+mF7inW2WlkfIKs9dkTOo80RAZNYvrShoSpZDQ/nAEy7toRh2bpkqM6sRheZFT5U3o44VnF8YKiz+E1fkP6Ppl8Uc/1ZCXdSHyZaJ61nmI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232754; c=relaxed/simple; bh=EVF+GTWRRa4IiloDcCWCijgTOo2QAa/+19abAkf0b+c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lmonTIeSG/IpytUHbETo690DyPWk/kdZIfZWMZM0zmzV3q2fXr2meQ7dote7NiuPzONJfp14fehGKTT/31/XSrmqvn35xj7SLuKfBZiR2ab2ct6WJbJbbA1Q67Hm4b74VFAyNLiXvERTLs+RMokJ743LSkoOViY3s4DlbTjqYUk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=bjfZCh4U; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="bjfZCh4U" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48D4hPck009974; Fri, 13 Sep 2024 13:05:51 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=GfF1GseNxNmkt 16Ag2Kldh2DY6cuIObZVMznUa/oTVY=; b=bjfZCh4U5DMXw7LGFblCpI1dWCh4t ghR9q5JLf7HDPxPzKqRF0p9mWpg23ZldINC8ctMtuOIPc1FW287EHAKAZ38Gii2G e2Winf49V3+064nM+BZkln8Qz6UiM2G9Zm+OAe1cdKoulz3BwiuuwaMEDEqqXy4U 5KRoT0a7kLoXMlUW4yaIv40Y4vJEGtTQel/N9Mxki3tJ0cwdLzBHJt9G0AWS1InR tCBCKvtDUsW/RjLYmbla6lE/5velgqeSPCxKBObWorZlz9IMC3RvToSvxW06QAA5 MZ40EQ4i+hAFBqoPjjMhnG05sl6BtpLXCZKRp9HiuAnhxE9FMai+5ivYQ== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41gegxa4hj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:50 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48DAN7Y6027315; Fri, 13 Sep 2024 13:05:50 GMT Received: from smtprelay01.fra02v.mail.ibm.com ([9.218.2.227]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 41h3v3nxuu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:49 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay01.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5kn436372760 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:46 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8D45920040; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4784A20043; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 6/7] s390/vdso: Move vdso symbol handling to separate header file Date: Fri, 13 Sep 2024 15:05:42 +0200 Message-ID: <20240913130544.2398678-7-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: yHpXC42-ulIeCzViaUjVIcceMkaX2wrx X-Proofpoint-ORIG-GUID: yHpXC42-ulIeCzViaUjVIcceMkaX2wrx X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_10,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=933 lowpriorityscore=0 suspectscore=0 priorityscore=1501 bulkscore=0 adultscore=0 impostorscore=0 clxscore=1015 malwarescore=0 mlxscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130091 The vdso.h header file, which is included at many places, includes generated header files. This can easily lead to recursive header file inclusions if the vdso code is changed. Therefore move the vdso symbol code, which requires the generated header files, to a separate header file, and include it at the two locations which require it. Signed-off-by: Heiko Carstens --- arch/s390/include/asm/vdso-symbols.h | 17 +++++++++++++++++ arch/s390/include/asm/vdso.h | 12 ------------ arch/s390/kernel/compat_signal.c | 2 +- arch/s390/kernel/signal.c | 2 +- 4 files changed, 19 insertions(+), 14 deletions(-) create mode 100644 arch/s390/include/asm/vdso-symbols.h diff --git a/arch/s390/include/asm/vdso-symbols.h b/arch/s390/include/asm/vdso-symbols.h new file mode 100644 index 000000000000..0df17574d788 --- /dev/null +++ b/arch/s390/include/asm/vdso-symbols.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __S390_VDSO_SYMBOLS_H__ +#define __S390_VDSO_SYMBOLS_H__ + +#include +#ifdef CONFIG_COMPAT +#include +#endif + +#define VDSO64_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso64_offset_##name)) +#ifdef CONFIG_COMPAT +#define VDSO32_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso32_offset_##name)) +#else +#define VDSO32_SYMBOL(tsk, name) (-1UL) +#endif + +#endif /* __S390_VDSO_SYMBOLS_H__ */ diff --git a/arch/s390/include/asm/vdso.h b/arch/s390/include/asm/vdso.h index 53165aa7813a..91061f0279be 100644 --- a/arch/s390/include/asm/vdso.h +++ b/arch/s390/include/asm/vdso.h @@ -6,18 +6,6 @@ #ifndef __ASSEMBLY__ -#include -#ifdef CONFIG_COMPAT -#include -#endif - -#define VDSO64_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso64_offset_##name)) -#ifdef CONFIG_COMPAT -#define VDSO32_SYMBOL(tsk, name) ((tsk)->mm->context.vdso_base + (vdso32_offset_##name)) -#else -#define VDSO32_SYMBOL(tsk, name) (-1UL) -#endif - extern struct vdso_data *vdso_data; int vdso_getcpu_init(void); diff --git a/arch/s390/kernel/compat_signal.c b/arch/s390/kernel/compat_signal.c index 1942e2a9f8db..5a86b9d1da71 100644 --- a/arch/s390/kernel/compat_signal.c +++ b/arch/s390/kernel/compat_signal.c @@ -24,11 +24,11 @@ #include #include #include +#include #include #include #include #include -#include #include #include "compat_linux.h" #include "compat_ptrace.h" diff --git a/arch/s390/kernel/signal.c b/arch/s390/kernel/signal.c index 6c2cb345402f..e48013cd832c 100644 --- a/arch/s390/kernel/signal.c +++ b/arch/s390/kernel/signal.c @@ -30,9 +30,9 @@ #include #include #include +#include #include #include -#include #include "entry.h" /* From patchwork Fri Sep 13 13:05:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heiko Carstens X-Patchwork-Id: 829471 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 183321DB525; Fri, 13 Sep 2024 13:05:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232755; cv=none; b=JVJh73ZA6BXM9SeU14R5djQOnb5mVZy4rti7RPkubfedrkkItqSYggDLn0xuJU4ni0Boyorm40an1MMrnqjitNazA5/cub67X7zy9jXEgqHNa1wsrwzIyT5lCQfVQyxgFMd8wm7gpYRpCdnNC1n5YUPW4yF4Bi0qx44YCscLNc4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726232755; c=relaxed/simple; bh=Hvq74IJKuTL2xHYEpZP2WVdFbo7BrLFFMZAacfNziqo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XsykiBHlarsC26Hhqvae3eIiemyY5UilVt9nNpVJqqJh9P9dnGO/ZeM48edvHmbrzE5Bj7m+6V9ZHUAq1c/4eTfizJnzh3yCiGJa5e8uuzJAUMW5X32ysmLrAQFAix+USvQouxqGj1wQuVeacigRyll78Sh2V0qcbGSzgaiQtxk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=YfgKJfd7; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="YfgKJfd7" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 48D73Pjl000999; Fri, 13 Sep 2024 13:05:52 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from :to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; s=pp1; bh=m7CQeHPkblrP9 mMZeWG5GnosbjlA6zZDSH1NJ2A+Jcw=; b=YfgKJfd70eUPOua07aPvBFlOpLvUN T6bk9XpfQfa9aD8yTfSldFfRT3J6UeK5SLatZ7WD2hQ/IbPnm9JfF3iQErZ0EXKu CiLp13ZcRoWy+vOECAbc+345ggTbv+aWKer34sgeLWuX3OTdTrAtuDLwaNDdZwkJ +EoUBV+kroi5DrI0SyglEztPQFle/4QVnoGqYmIEO+zOKwfayyqgGsqybpxGlQGj zUojADWpMcYJ9xRgPLZ1WgzZHRhjulRWJAefQ2mswOs1iRvrALsGoxNTrWNt4RME sCs52X6maskvbeUa5FyZS2BqPIJ5/XDjIZDjpf5gP9RDCkymm1lCqn2eg== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 41geg01ugn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:51 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 48D9u9kC027297; Fri, 13 Sep 2024 13:05:50 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 41h3v3nxuy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 13 Sep 2024 13:05:50 +0000 Received: from smtpav07.fra02v.mail.ibm.com (smtpav07.fra02v.mail.ibm.com [10.20.54.106]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 48DD5lN649807798 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 13 Sep 2024 13:05:47 GMT Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EC79F20040; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) Received: from smtpav07.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 979512004B; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by smtpav07.fra02v.mail.ibm.com (Postfix) with ESMTP; Fri, 13 Sep 2024 13:05:46 +0000 (GMT) From: Heiko Carstens To: "Jason A . Donenfeld" Cc: Alexander Gordeev , Vasily Gorbik , Christian Borntraeger , Sven Schnelle , Harald Freudenberger , Stefan Liebler , linux-kernel@vger.kernel.org, linux-crypto@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH 7/7] s390/vdso: Wire up getrandom() vdso implementation Date: Fri, 13 Sep 2024 15:05:43 +0200 Message-ID: <20240913130544.2398678-8-hca@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240913130544.2398678-1-hca@linux.ibm.com> References: <20240913130544.2398678-1-hca@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: zLiDSlZ9YgD4_5zJVgBAV7Irvhj0xw_d X-Proofpoint-ORIG-GUID: zLiDSlZ9YgD4_5zJVgBAV7Irvhj0xw_d X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1039,Hydra:6.0.680,FMLib:17.12.60.29 definitions=2024-09-13_09,2024-09-13_02,2024-09-02_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxscore=0 malwarescore=0 lowpriorityscore=0 suspectscore=0 mlxlogscore=999 priorityscore=1501 adultscore=0 clxscore=1015 spamscore=0 bulkscore=0 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2408220000 definitions=main-2409130087 Provide the s390 specific vdso getrandom() architecture backend. _vdso_rng_data required data is placed within the _vdso_data vvar page, by using a hardcoded offset larger than vdso_data. As required the chacha20 implementation does not write to the stack. The implementation follows more or less the arm64 implementations and makes use of vector instructions. It has a fallback to the getrandom() system call for machines where the vector facility is not installed. The check if the vector facility is installed, as well as an optimization for machines with the vector-enhancements facility 2, is implemented with alternatives, avoiding runtime checks. Note that __kernel_getrandom() is implemented without the vdso user wrapper which would setup a stack frame for odd cases (aka very old glibc variants) where the caller has not done that. All callers of __kernel_getrandom() are required to setup a stack frame, like the C ABI requires it. The vdso testcases vdso_test_getrandom and vdso_test_chacha pass. Signed-off-by: Heiko Carstens --- arch/s390/Kconfig | 1 + arch/s390/include/asm/fpu-insn-asm.h | 22 +++ arch/s390/include/asm/vdso/getrandom.h | 40 +++++ arch/s390/include/asm/vdso/vsyscall.h | 15 ++ arch/s390/kernel/vdso.c | 7 +- arch/s390/kernel/vdso64/Makefile | 9 +- arch/s390/kernel/vdso64/vdso.h | 1 + arch/s390/kernel/vdso64/vdso64.lds.S | 3 + arch/s390/kernel/vdso64/vgetrandom-chacha.S | 184 ++++++++++++++++++++ arch/s390/kernel/vdso64/vgetrandom.c | 14 ++ tools/arch/s390/vdso | 1 + tools/testing/selftests/vDSO/Makefile | 2 +- 12 files changed, 290 insertions(+), 9 deletions(-) create mode 100644 arch/s390/include/asm/vdso/getrandom.h create mode 100644 arch/s390/kernel/vdso64/vgetrandom-chacha.S create mode 100644 arch/s390/kernel/vdso64/vgetrandom.c create mode 120000 tools/arch/s390/vdso diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index c60e699e99f5..b0d0b3a8d196 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -243,6 +243,7 @@ config S390 select TRACE_IRQFLAGS_SUPPORT select TTY select USER_STACKTRACE_SUPPORT + select VDSO_GETRANDOM select VIRT_CPU_ACCOUNTING select ZONE_DMA # Note: keep the above list sorted alphabetically diff --git a/arch/s390/include/asm/fpu-insn-asm.h b/arch/s390/include/asm/fpu-insn-asm.h index 02ccfe46050a..d296322be4bc 100644 --- a/arch/s390/include/asm/fpu-insn-asm.h +++ b/arch/s390/include/asm/fpu-insn-asm.h @@ -407,6 +407,28 @@ MRXBOPC 0, 0x0E, v1 .endm +/* VECTOR STORE BYTE REVERSED ELEMENTS */ + .macro VSTBR vr1, disp, index="%r0", base, m + VX_NUM v1, \vr1 + GR_NUM x2, \index + GR_NUM b2, \base + .word 0xE600 | ((v1&15) << 4) | (x2&15) + .word (b2 << 12) | (\disp) + MRXBOPC \m, 0x0E, v1 +.endm +.macro VSTBRH vr1, disp, index="%r0", base + VSTBR \vr1, \disp, \index, \base, 1 +.endm +.macro VSTBRF vr1, disp, index="%r0", base + VSTBR \vr1, \disp, \index, \base, 2 +.endm +.macro VSTBRG vr1, disp, index="%r0", base + VSTBR \vr1, \disp, \index, \base, 3 +.endm +.macro VSTBRQ vr1, disp, index="%r0", base + VSTBR \vr1, \disp, \index, \base, 4 +.endm + /* VECTOR STORE MULTIPLE */ .macro VSTM vfrom, vto, disp, base, hint=3 VX_NUM v1, \vfrom diff --git a/arch/s390/include/asm/vdso/getrandom.h b/arch/s390/include/asm/vdso/getrandom.h new file mode 100644 index 000000000000..36355af7160b --- /dev/null +++ b/arch/s390/include/asm/vdso/getrandom.h @@ -0,0 +1,40 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_VDSO_GETRANDOM_H +#define __ASM_VDSO_GETRANDOM_H + +#ifndef __ASSEMBLY__ + +#include +#include +#include +#include +#include + +/** + * getrandom_syscall - Invoke the getrandom() syscall. + * @buffer: Destination buffer to fill with random bytes. + * @len: Size of @buffer in bytes. + * @flags: Zero or more GRND_* flags. + * Returns: The number of random bytes written to @buffer, or a negative value indicating an error. + */ +static __always_inline ssize_t getrandom_syscall(void *buffer, size_t len, unsigned int flags) +{ + return syscall3(__NR_getrandom, (long)buffer, (long)len, (long)flags); +} + +static __always_inline const struct vdso_rng_data *__arch_get_vdso_rng_data(void) +{ + /* + * The RNG data is in the real VVAR data page, but if a task belongs to a time namespace + * then VVAR_DATA_PAGE_OFFSET points to the namespace-specific VVAR page and VVAR_TIMENS_ + * PAGE_OFFSET points to the real VVAR page. + */ + if (IS_ENABLED(CONFIG_TIME_NS) && _vdso_data->clock_mode == VDSO_CLOCKMODE_TIMENS) + return (void *)&_vdso_rng_data + VVAR_TIMENS_PAGE_OFFSET * PAGE_SIZE; + return &_vdso_rng_data; +} + +#endif /* !__ASSEMBLY__ */ + +#endif /* __ASM_VDSO_GETRANDOM_H */ diff --git a/arch/s390/include/asm/vdso/vsyscall.h b/arch/s390/include/asm/vdso/vsyscall.h index 6c67c08cefdd..3c5d5e47814e 100644 --- a/arch/s390/include/asm/vdso/vsyscall.h +++ b/arch/s390/include/asm/vdso/vsyscall.h @@ -2,12 +2,21 @@ #ifndef __ASM_VDSO_VSYSCALL_H #define __ASM_VDSO_VSYSCALL_H +#define __VDSO_RND_DATA_OFFSET 768 + #ifndef __ASSEMBLY__ #include #include #include #include + +enum vvar_pages { + VVAR_DATA_PAGE_OFFSET, + VVAR_TIMENS_PAGE_OFFSET, + VVAR_NR_PAGES +}; + /* * Update the vDSO data page to keep in sync with kernel timekeeping. */ @@ -18,6 +27,12 @@ static __always_inline struct vdso_data *__s390_get_k_vdso_data(void) } #define __arch_get_k_vdso_data __s390_get_k_vdso_data +static __always_inline struct vdso_rng_data *__s390_get_k_vdso_rnd_data(void) +{ + return (void *)vdso_data + __VDSO_RND_DATA_OFFSET; +} +#define __arch_get_k_vdso_rng_data __s390_get_k_vdso_rnd_data + /* The asm-generic header needs to be included after the definitions above */ #include diff --git a/arch/s390/kernel/vdso.c b/arch/s390/kernel/vdso.c index 8e4e6b316337..598b512cde01 100644 --- a/arch/s390/kernel/vdso.c +++ b/arch/s390/kernel/vdso.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -31,12 +32,6 @@ static union vdso_data_store vdso_data_store __page_aligned_data; struct vdso_data *vdso_data = vdso_data_store.data; -enum vvar_pages { - VVAR_DATA_PAGE_OFFSET, - VVAR_TIMENS_PAGE_OFFSET, - VVAR_NR_PAGES, -}; - #ifdef CONFIG_TIME_NS struct vdso_data *arch_get_vdso_data(void *vvar_page) { diff --git a/arch/s390/kernel/vdso64/Makefile b/arch/s390/kernel/vdso64/Makefile index ba19c0ca7c87..37bb4b761229 100644 --- a/arch/s390/kernel/vdso64/Makefile +++ b/arch/s390/kernel/vdso64/Makefile @@ -3,12 +3,17 @@ # Include the generic Makefile to check the built vdso. include $(srctree)/lib/vdso/Makefile -obj-vdso64 = vdso_user_wrapper.o note.o -obj-cvdso64 = vdso64_generic.o getcpu.o +obj-vdso64 = vdso_user_wrapper.o note.o vgetrandom-chacha.o +obj-cvdso64 = vdso64_generic.o getcpu.o vgetrandom.o VDSO_CFLAGS_REMOVE := -pg $(CC_FLAGS_FTRACE) $(CC_FLAGS_EXPOLINE) $(CC_FLAGS_CHECK_STACK) CFLAGS_REMOVE_getcpu.o = $(VDSO_CFLAGS_REMOVE) +CFLAGS_REMOVE_vgetrandom.o = $(VDSO_CFLAGS_REMOVE) CFLAGS_REMOVE_vdso64_generic.o = $(VDSO_CFLAGS_REMOVE) +ifneq ($(c-getrandom-y),) + CFLAGS_vgetrandom.o += -include $(c-getrandom-y) +endif + # Build rules targets := $(obj-vdso64) $(obj-cvdso64) vdso64.so vdso64.so.dbg diff --git a/arch/s390/kernel/vdso64/vdso.h b/arch/s390/kernel/vdso64/vdso.h index 34c7a2312f9d..9e5397e7b590 100644 --- a/arch/s390/kernel/vdso64/vdso.h +++ b/arch/s390/kernel/vdso64/vdso.h @@ -10,5 +10,6 @@ int __s390_vdso_getcpu(unsigned *cpu, unsigned *node, struct getcpu_cache *unuse int __s390_vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz); int __s390_vdso_clock_gettime(clockid_t clock, struct __kernel_timespec *ts); int __s390_vdso_clock_getres(clockid_t clock, struct __kernel_timespec *ts); +ssize_t __kernel_getrandom(void *buffer, size_t len, unsigned int flags, void *opaque_state, size_t opaque_len); #endif /* __ARCH_S390_KERNEL_VDSO64_VDSO_H */ diff --git a/arch/s390/kernel/vdso64/vdso64.lds.S b/arch/s390/kernel/vdso64/vdso64.lds.S index fa02c6ae3cac..753040a4b5ab 100644 --- a/arch/s390/kernel/vdso64/vdso64.lds.S +++ b/arch/s390/kernel/vdso64/vdso64.lds.S @@ -4,6 +4,7 @@ * library */ +#include #include #include @@ -13,6 +14,7 @@ OUTPUT_ARCH(s390:64-bit) SECTIONS { PROVIDE(_vdso_data = . - __VVAR_PAGES * PAGE_SIZE); + PROVIDE(_vdso_rng_data = _vdso_data + __VDSO_RND_DATA_OFFSET); #ifdef CONFIG_TIME_NS PROVIDE(_timens_data = _vdso_data + PAGE_SIZE); #endif @@ -144,6 +146,7 @@ VERSION __kernel_restart_syscall; __kernel_rt_sigreturn; __kernel_sigreturn; + __kernel_getrandom; local: *; }; } diff --git a/arch/s390/kernel/vdso64/vgetrandom-chacha.S b/arch/s390/kernel/vdso64/vgetrandom-chacha.S new file mode 100644 index 000000000000..ecd44cf0eaba --- /dev/null +++ b/arch/s390/kernel/vdso64/vgetrandom-chacha.S @@ -0,0 +1,184 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include +#include +#include + +#define STATE0 %v0 +#define STATE1 %v1 +#define STATE2 %v2 +#define STATE3 %v3 +#define COPY0 %v4 +#define COPY1 %v5 +#define COPY2 %v6 +#define COPY3 %v7 +#define PERM4 %v16 +#define PERM8 %v17 +#define PERM12 %v18 +#define BEPERM %v19 +#define TMP0 %v20 +#define TMP1 %v21 +#define TMP2 %v22 +#define TMP3 %v23 + + .section .rodata + + .balign 128 +.Lconstants: + .long 0x61707865,0x3320646e,0x79622d32,0x6b206574 # endian-neutral + .long 0x04050607,0x08090a0b,0x0c0d0e0f,0x00010203 # rotl 4 bytes + .long 0x08090a0b,0x0c0d0e0f,0x00010203,0x04050607 # rotl 8 bytes + .long 0x0c0d0e0f,0x00010203,0x04050607,0x08090a0b # rotl 12 bytes + .long 0x03020100,0x07060504,0x0b0a0908,0x0f0e0d0c # byte swap + + .text +/* + * s390 ChaCha20 implementation meant for vDSO. Produces a given positive + * number of blocks of output with nonce 0, taking an input key and 8-bytes + * counter. Does not spill to the stack. + * + * void __arch_chacha20_blocks_nostack(uint8_t *dst_bytes, + * const uint8_t *key, + * uint32_t *counter, + * size_t nblocks) + */ +SYM_FUNC_START(__arch_chacha20_blocks_nostack) + larl %r1,.Lconstants + + /* COPY0 = "expand 32-byte k" */ + VL COPY0,0,,%r1 + + /* PERM4-PERM12,BEPERM = byte selectors for VPERM */ + VLM PERM4,BEPERM,16,%r1 + + /* COPY1,COPY2 = key */ + VLM COPY1,COPY2,0,%r3 + + /* COPY3 = counter || zero nonce */ + lg %r3,0(%r4) + VZERO COPY3 + VLVGG COPY3,%r3,0 + + lghi %r1,0 +.Lblock: + VLR STATE0,COPY0 + VLR STATE1,COPY1 + VLR STATE2,COPY2 + VLR STATE3,COPY3 + + lghi %r0,10 +.Ldoubleround: + /* STATE0 += STATE1, STATE3 = rotl32(STATE3 ^ STATE0, 16) */ + VAF STATE0,STATE0,STATE1 + VX STATE3,STATE3,STATE0 + VERLLF STATE3,STATE3,16 + + /* STATE2 += STATE3, STATE1 = rotl32(STATE1 ^ STATE2, 12) */ + VAF STATE2,STATE2,STATE3 + VX STATE1,STATE1,STATE2 + VERLLF STATE1,STATE1,12 + + /* STATE0 += STATE1, STATE3 = rotl32(STATE3 ^ STATE0, 8) */ + VAF STATE0,STATE0,STATE1 + VX STATE3,STATE3,STATE0 + VERLLF STATE3,STATE3,8 + + /* STATE2 += STATE3, STATE1 = rotl32(STATE1 ^ STATE2, 7) */ + VAF STATE2,STATE2,STATE3 + VX STATE1,STATE1,STATE2 + VERLLF STATE1,STATE1,7 + + /* STATE1[0,1,2,3] = STATE1[1,2,3,0] */ + VPERM STATE1,STATE1,STATE1,PERM4 + /* STATE2[0,1,2,3] = STATE2[2,3,0,1] */ + VPERM STATE2,STATE2,STATE2,PERM8 + /* STATE3[0,1,2,3] = STATE3[3,0,1,2] */ + VPERM STATE3,STATE3,STATE3,PERM12 + + /* STATE0 += STATE1, STATE3 = rotl32(STATE3 ^ STATE0, 16) */ + VAF STATE0,STATE0,STATE1 + VX STATE3,STATE3,STATE0 + VERLLF STATE3,STATE3,16 + + /* STATE2 += STATE3, STATE1 = rotl32(STATE1 ^ STATE2, 12) */ + VAF STATE2,STATE2,STATE3 + VX STATE1,STATE1,STATE2 + VERLLF STATE1,STATE1,12 + + /* STATE0 += STATE1, STATE3 = rotl32(STATE3 ^ STATE0, 8) */ + VAF STATE0,STATE0,STATE1 + VX STATE3,STATE3,STATE0 + VERLLF STATE3,STATE3,8 + + /* STATE2 += STATE3, STATE1 = rotl32(STATE1 ^ STATE2, 7) */ + VAF STATE2,STATE2,STATE3 + VX STATE1,STATE1,STATE2 + VERLLF STATE1,STATE1,7 + + /* STATE1[0,1,2,3] = STATE1[3,0,1,2] */ + VPERM STATE1,STATE1,STATE1,PERM12 + /* STATE2[0,1,2,3] = STATE2[2,3,0,1] */ + VPERM STATE2,STATE2,STATE2,PERM8 + /* STATE3[0,1,2,3] = STATE3[1,2,3,0] */ + VPERM STATE3,STATE3,STATE3,PERM4 + brctg %r0,.Ldoubleround + + /* OUTPUT0 = STATE0 + STATE0 */ + VAF STATE0,STATE0,COPY0 + /* OUTPUT1 = STATE1 + STATE1 */ + VAF STATE1,STATE1,COPY1 + /* OUTPUT2 = STATE2 + STATE2 */ + VAF STATE2,STATE2,COPY2 + /* OUTPUT2 = STATE3 + STATE3 */ + VAF STATE3,STATE3,COPY3 + + /* + * 32 bit wise little endian store to OUTPUT. If the vector + * enhancement facility 2 is not installed use the slow path. + */ + ALTERNATIVE "brc 0xf,.Lstoreslow", "nop", ALT_FACILITY(148) + VSTBRF STATE0,0,,%r2 + VSTBRF STATE1,16,,%r2 + VSTBRF STATE2,32,,%r2 + VSTBRF STATE3,48,,%r2 +.Lstoredone: + + /* ++COPY3.COUNTER */ + alsih %r3,1 + alcr %r3,%r1 + VLVGG COPY3,%r3,0 + + /* OUTPUT += 64, --NBLOCKS */ + aghi %r2,64 + brctg %r5,.Lblock + + /* COUNTER = COPY3.COUNTER */ + stg %r3,0(%r4) + + /* Zero out potentially sensitive regs */ + VZERO STATE0 + VZERO STATE1 + VZERO STATE2 + VZERO STATE3 + VZERO COPY1 + VZERO COPY2 + + /* Early exit if TMP0-TMP3 have not been used */ + ALTERNATIVE "nopr", "br %r14", ALT_FACILITY(148) + + VZERO TMP0 + VZERO TMP1 + VZERO TMP2 + VZERO TMP3 + + br %r14 + +.Lstoreslow: + /* Convert STATE to little endian format and store to OUTPUT */ + VPERM TMP0,STATE0,STATE0,BEPERM + VPERM TMP1,STATE1,STATE1,BEPERM + VPERM TMP2,STATE2,STATE2,BEPERM + VPERM TMP3,STATE3,STATE3,BEPERM + VSTM TMP0,TMP3,0,%r2 + j .Lstoredone +SYM_FUNC_END(__arch_chacha20_blocks_nostack) diff --git a/arch/s390/kernel/vdso64/vgetrandom.c b/arch/s390/kernel/vdso64/vgetrandom.c new file mode 100644 index 000000000000..b5268b507fb5 --- /dev/null +++ b/arch/s390/kernel/vdso64/vgetrandom.c @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include +#include +#include "vdso.h" + +ssize_t __kernel_getrandom(void *buffer, size_t len, unsigned int flags, void *opaque_state, size_t opaque_len) +{ + if (test_facility(129)) + return __cvdso_getrandom(buffer, len, flags, opaque_state, opaque_len); + if (unlikely(opaque_len == ~0UL && !buffer && !len && !flags)) + return -ENOSYS; + return getrandom_syscall(buffer, len, flags); +} diff --git a/tools/arch/s390/vdso b/tools/arch/s390/vdso new file mode 120000 index 000000000000..6cf4c1cebdcd --- /dev/null +++ b/tools/arch/s390/vdso @@ -0,0 +1 @@ +../../../arch/s390/kernel/vdso64 \ No newline at end of file diff --git a/tools/testing/selftests/vDSO/Makefile b/tools/testing/selftests/vDSO/Makefile index 86ebc4115eda..af9cedbf5357 100644 --- a/tools/testing/selftests/vDSO/Makefile +++ b/tools/testing/selftests/vDSO/Makefile @@ -9,7 +9,7 @@ ifeq ($(ARCH),$(filter $(ARCH),x86 x86_64)) TEST_GEN_PROGS += vdso_standalone_test_x86 endif TEST_GEN_PROGS += vdso_test_correctness -ifeq ($(ARCH)$(CONFIG_X86_32),$(filter $(ARCH)$(CONFIG_X86_32),x86 x86_64 loongarch arm64 powerpc)) +ifeq ($(ARCH)$(CONFIG_X86_32),$(filter $(ARCH)$(CONFIG_X86_32),x86 x86_64 loongarch arm64 powerpc s390)) TEST_GEN_PROGS += vdso_test_getrandom TEST_GEN_PROGS += vdso_test_chacha endif