From patchwork Mon Aug 24 18:36:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Fainelli X-Patchwork-Id: 248239 Delivered-To: patch@linaro.org Received: by 2002:a17:906:b105:0:0:0:0 with SMTP id u5csp1893907ejy; Mon, 24 Aug 2020 11:36:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz9IyODzZPt5m4kBeY+OvZKzAVs+0+sKObGiMBnSexpuaLqGpunCFv5aZLvF9JRxKFA45oG X-Received: by 2002:a17:906:925a:: with SMTP id c26mr6651464ejx.121.1598294194039; Mon, 24 Aug 2020 11:36:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1598294194; cv=none; d=google.com; s=arc-20160816; b=TvXapMReAAf0w91y4Fc8bQ99Li5cMUPkkMZsY5QkisTYIkrO/rCQOZ/OFxWZVqjNZQ 48Pa2Yb5UHD4woqBl9XlqoNb7iKI1EN+h/FSaSTS73qVB71QDMzTOy7ouqgeJPLyJIey w7IeR+l9capysrAP308YPi7C7V8K3tOZiJKJN3JydKs9PP/FAke/05+rXRmZUU/Ge1HK zdfNWSzxXDT4ocnQ1imwq6N6BvSM2WDwhc7kqkIRaVOCYDEDSiSwAqLrA9nnGZWq7EN5 X3m+i7ufiUd57Up1VbD/VWyR4jA6bQMfFtJMYiWjeEsV4OECmbSaZyEXpvJndsW5HDM+ zKwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=b96xElma2wVNkZLWPyInMIgVAkPIrOZeVr/+MC3kMPQ=; b=rLvz0GuGwGnUsNZxHQVjSH7zUOktxS2047bKbDAoPkcZhLAbuZNDeSui10l2ESsvd2 kME+cvLqwG3WGkg2pvWWZh5lNPFcGbMd2SM/Yb3T8BxYlILHuIEv34NxjW8hRdTUITIc Kq/mCFY9N+EMJOU67/Nf8DjjbPrQTvPdbZlk052S+FAEVfne3vhAUf50iCBBDrMAGBqV rOGlO20H0CbBW69Jj0VwaToOv8FSAVXGo9tT7jGk3oVG7m7Q65XIMuYa0gfHOY21ZAUF HbWeNTYjS5gNwDrgrAaKow2qEQrIRq7kR8glPf8Eb1dfxgJ3xIyzV5qkQTSRYKcTqewZ 0exw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=kJIDOILb; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id n16si3766487eds.250.2020.08.24.11.36.33; Mon, 24 Aug 2020 11:36:34 -0700 (PDT) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=kJIDOILb; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727036AbgHXSgc (ORCPT + 15 others); Mon, 24 Aug 2020 14:36:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727001AbgHXSg2 (ORCPT ); Mon, 24 Aug 2020 14:36:28 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CBD98C061573; Mon, 24 Aug 2020 11:36:27 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id w186so2875636pgb.8; Mon, 24 Aug 2020 11:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=b96xElma2wVNkZLWPyInMIgVAkPIrOZeVr/+MC3kMPQ=; b=kJIDOILbYl/7UguWLWS2J4Ts2X2NdD8kL8uqUY5o65T9vBehdn1gGl4vgizV6s8rVE B2euk//REx4SjEjprw+qtTl8QzH2yI3G+JTE8e8gSkGADHzc+Qp1k3ITuesmITUK27rz Vl7l62szu+Mw3v3Kyp70IdshpPjbRfTJSi6m4R5vCr/ZYlA3/78lFUYrscQMWK4BOWwM WM/D6Izu1r26KTHmwEK63H2D5yTTyQFZ1px05D97BaEmM8qZOPNF8bcM4hsj2067WCL/ PZ71fawRO5V1MVoC26lU2gTUznvnvnlCOV3v5OiM1KDpNGTsumW1VKRC/Q4/KYFhSDAJ BVYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=b96xElma2wVNkZLWPyInMIgVAkPIrOZeVr/+MC3kMPQ=; b=nLTCgPU0aBRS2SsvZH0XioKcq6bSrvs4z9RpY40utFEgAa60sbwkmMuN1srGm8LTgz tDGFwrXSSGTzxisoMpiBr4/lTYoerWusQnTheN5ZCBWoeZmuZxQolQhcQnEv696a5wg7 5kbLS29nEEGdB9sP2Lw6CKaioPvptiLtHxLJZNg6WtiykupWAI2B8UVOTfJMcurrxSQH cb5t9SkjW/ef0/9UbGlncXqkBfkOxS9Tet3npWQkfgEXMxiW2BG7ZJEtexyXN6+OiD1R TfyrmpBYHr827qrb7H+M1kc/aCL7a9mFd9dBY7+eWlT5ins42K3N30baeEjgSnssn0+c fRWw== X-Gm-Message-State: AOAM530OdVxkb7FIXTe9s0BhsNaMgo6hpVQH00eRCRfq2kkTaAEK/MY+ 1BRgeGb6Hsu5RDt+BWR6mLQ= X-Received: by 2002:a63:f44d:: with SMTP id p13mr4113885pgk.363.1598294187273; Mon, 24 Aug 2020 11:36:27 -0700 (PDT) Received: from stbirv-lnx-3.igp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id u191sm10235065pgu.56.2020.08.24.11.36.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Aug 2020 11:36:26 -0700 (PDT) From: Florian Fainelli To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Florian Fainelli , Greg KH , Will Deacon , stable@vger.kernel.org, Catalin Marinas , Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , Mark Rutland , Mark Brown , Kristina Martsenko , Sami Tolvanen , Andrew Jones , Ard Biesheuvel , Nick Desaulniers , Fangrui Song , open list , "open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)" Subject: [PATCH stable 4.9 v3 1/2] arm64: Add support for SB barrier and patch in over DSB; ISB sequences Date: Mon, 24 Aug 2020 11:36:21 -0700 Message-Id: <1598294182-34012-2-git-send-email-f.fainelli@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1598294182-34012-1-git-send-email-f.fainelli@gmail.com> References: <1598294182-34012-1-git-send-email-f.fainelli@gmail.com> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit bd4fb6d270bc423a9a4098108784f7f9254c4e6d upstream We currently use a DSB; ISB sequence to inhibit speculation in set_fs(). Whilst this works for current CPUs, future CPUs may implement a new SB barrier instruction which acts as an architected speculation barrier. On CPUs that support it, patch in an SB; NOP sequence over the DSB; ISB sequence and advertise the presence of the new instruction to userspace. Signed-off-by: Will Deacon [florian: - adjust number capabilities - wire decoding of ftr_id_aa64_isar1 - __emit_inst -> __inst_arm - sys_insn -> sys_reg] Signed-off-by: Florian Fainelli --- arch/arm64/include/asm/assembler.h | 13 +++++++++++++ arch/arm64/include/asm/barrier.h | 4 ++++ arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/sysreg.h | 13 +++++++++++++ arch/arm64/include/asm/uaccess.h | 3 +-- arch/arm64/include/uapi/asm/hwcap.h | 1 + arch/arm64/kernel/cpufeature.c | 22 +++++++++++++++++++++- arch/arm64/kernel/cpuinfo.c | 1 + 8 files changed, 56 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 3f85bbcd7e40..0f689aa0b300 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -95,6 +95,19 @@ .endm /* + * Speculation barrier + */ + .macro sb +alternative_if_not ARM64_HAS_SB + dsb nsh + isb +alternative_else + SB_BARRIER_INSN + nop +alternative_endif + .endm + +/* * Sanitise a 64-bit bounded index wrt speculation, returning zero if out * of bounds. */ diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index 0b0755c961ac..159329160fb4 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -33,6 +33,10 @@ #define csdb() asm volatile("hint #20" : : : "memory") +#define spec_bar() asm volatile(ALTERNATIVE("dsb nsh\nisb\n", \ + SB_BARRIER_INSN"nop\n", \ + ARM64_HAS_SB)) + #define mb() dsb(sy) #define rmb() dsb(ld) #define wmb() dsb(st) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 8c7c4b23a8b1..8a25f6e39f3a 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -38,7 +38,8 @@ #define ARM64_HARDEN_BRANCH_PREDICTOR 17 #define ARM64_SSBD 18 #define ARM64_MISMATCHED_CACHE_TYPE 19 +#define ARM64_HAS_SB 20 -#define ARM64_NCAPS 20 +#define ARM64_NCAPS 21 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 88bbe364b6ae..1c49eaac1938 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -86,6 +86,12 @@ #define SET_PSTATE_UAO(x) __inst_arm(0xd5000000 | REG_PSTATE_UAO_IMM |\ (!!x)<<8 | 0x1f) +#define __SYS_BARRIER_INSN(CRm, op2, Rt) \ + __inst_arm(0xd5000000 | sys_reg(0, 3, 3, (CRm), (op2)) | ((Rt) & 0x1f)) + +#define SB_BARRIER_INSN __SYS_BARRIER_INSN(0, 7, 31) + + /* Common SCTLR_ELx flags. */ #define SCTLR_ELx_EE (1 << 25) #define SCTLR_ELx_I (1 << 12) @@ -116,6 +122,13 @@ #define ID_AA64ISAR0_SHA1_SHIFT 8 #define ID_AA64ISAR0_AES_SHIFT 4 +/* id_aa64isar1 */ +#define ID_AA64ISAR1_SB_SHIFT 36 +#define ID_AA64ISAR1_LRCPC_SHIFT 20 +#define ID_AA64ISAR1_FCMA_SHIFT 16 +#define ID_AA64ISAR1_JSCVT_SHIFT 12 +#define ID_AA64ISAR1_DPB_SHIFT 0 + /* id_aa64pfr0 */ #define ID_AA64PFR0_CSV3_SHIFT 60 #define ID_AA64PFR0_CSV2_SHIFT 56 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index f5cd96c60eb9..195323089e5a 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -71,8 +71,7 @@ static inline void set_fs(mm_segment_t fs) * Prevent a mispredicted conditional call to set_fs from forwarding * the wrong address limit to access_ok under speculation. */ - dsb(nsh); - isb(); + spec_bar(); /* * Enable/disable UAO so that copy_to_user() etc can access diff --git a/arch/arm64/include/uapi/asm/hwcap.h b/arch/arm64/include/uapi/asm/hwcap.h index a739287ef6a3..3aa659624968 100644 --- a/arch/arm64/include/uapi/asm/hwcap.h +++ b/arch/arm64/include/uapi/asm/hwcap.h @@ -30,5 +30,6 @@ #define HWCAP_ATOMICS (1 << 8) #define HWCAP_FPHP (1 << 9) #define HWCAP_ASIMDHP (1 << 10) +#define HWCAP_SB (1 << 29) #endif /* _UAPI__ASM_HWCAP_H */ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 8cf001baee21..1a5879c0b711 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -93,6 +93,15 @@ static const struct arm64_ftr_bits ftr_id_aa64isar0[] = { ARM64_FTR_END, }; +static const struct arm64_ftr_bits ftr_id_aa64isar1[] = { + ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_SB_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_LRCPC_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_FCMA_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_JSCVT_SHIFT, 4, 0), + ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64ISAR1_DPB_SHIFT, 4, 0), + ARM64_FTR_END, +}; + static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { ARM64_FTR_BITS(FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV3_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0), @@ -326,7 +335,7 @@ static const struct __ftr_reg_entry { /* Op1 = 0, CRn = 0, CRm = 6 */ ARM64_FTR_REG(SYS_ID_AA64ISAR0_EL1, ftr_id_aa64isar0), - ARM64_FTR_REG(SYS_ID_AA64ISAR1_EL1, ftr_aa64raz), + ARM64_FTR_REG(SYS_ID_AA64ISAR1_EL1, ftr_id_aa64isar1), /* Op1 = 0, CRn = 0, CRm = 7 */ ARM64_FTR_REG(SYS_ID_AA64MMFR0_EL1, ftr_id_aa64mmfr0), @@ -949,6 +958,16 @@ static const struct arm64_cpu_capabilities arm64_features[] = { .enable = kpti_install_ng_mappings, }, #endif + { + .desc = "Speculation barrier (SB)", + .capability = ARM64_HAS_SB, + .def_scope = SCOPE_SYSTEM, + .matches = has_cpuid_feature, + .sys_reg = SYS_ID_AA64ISAR1_EL1, + .field_pos = ID_AA64ISAR1_SB_SHIFT, + .sign = FTR_UNSIGNED, + .min_field_value = 1, + }, {}, }; @@ -976,6 +995,7 @@ static const struct arm64_cpu_capabilities arm64_elf_hwcaps[] = { HWCAP_CAP(SYS_ID_AA64PFR0_EL1, ID_AA64PFR0_FP_SHIFT, FTR_SIGNED, 1, CAP_HWCAP, HWCAP_FPHP), HWCAP_CAP(SYS_ID_AA64PFR0_EL1, ID_AA64PFR0_ASIMD_SHIFT, FTR_SIGNED, 0, CAP_HWCAP, HWCAP_ASIMD), HWCAP_CAP(SYS_ID_AA64PFR0_EL1, ID_AA64PFR0_ASIMD_SHIFT, FTR_SIGNED, 1, CAP_HWCAP, HWCAP_ASIMDHP), + HWCAP_CAP(SYS_ID_AA64ISAR1_EL1, ID_AA64ISAR1_SB_SHIFT, FTR_UNSIGNED, 1, CAP_HWCAP, HWCAP_SB), {}, }; diff --git a/arch/arm64/kernel/cpuinfo.c b/arch/arm64/kernel/cpuinfo.c index b3d5b3e8fbcb..5de2d55a999e 100644 --- a/arch/arm64/kernel/cpuinfo.c +++ b/arch/arm64/kernel/cpuinfo.c @@ -63,6 +63,7 @@ static const char *const hwcap_str[] = { "atomics", "fphp", "asimdhp", + "sb", NULL }; From patchwork Mon Aug 24 18:36:22 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Florian Fainelli X-Patchwork-Id: 248240 Delivered-To: patch@linaro.org Received: by 2002:a17:906:b105:0:0:0:0 with SMTP id u5csp1894008ejy; Mon, 24 Aug 2020 11:36:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzHCg05tWOdi9IkfxoxD+W/yJ0sx9NRnB1CoygOyo25Ynwr6WC3ibnHxHN1qKkjCC8Gap3n X-Received: by 2002:a50:ed02:: with SMTP id j2mr4594669eds.41.1598294205552; Mon, 24 Aug 2020 11:36:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1598294205; cv=none; d=google.com; s=arc-20160816; b=uhr2+BU9m/PbTrX9DKAOKrvDFmiZcDjOHAI+aDPDUVneTbsm3pfzoETNhnaTvxxDd+ mvQYDpxSF2wC6fsOavDG4rt4/6DcXKrM1skhBGUASVcog2BM94wGW4VokpTQEPslFMFl BPvlWPDEsCRxV6okgiK59slnkNFy7KcptROiulC6F7w3QDrFEknLFhRYpSRs/Fe7OoHA ZHKyAFZBXspjBdkqRWQSaUi/PEyHAnSea7zLEQxn4PQIUjDBmQwtKTCRRxKBEnHzSEyo B0tg0DZPa3EICgkzRJQvpm+PG6+AQxblCQj3tq1L7Ldlv+pzS4LHR46CIKr7HnNdfB7t d/tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=3q7TgN3lxewyLqAIBfP9LryeeItLVKAQXHJvLwyIaWM=; b=SHfIp1artsfbMRGvU8IVUYJHLdx++ZlbLR7BYznsvDOKggsreILaPobUknYdQLh4Kh bTzlJK4PI+d62YEGgnAiVHZ6zQNvhEZh9J0sRunTJ84DsdmCifBc/Ccme1ap7CLeF2f5 IHl1m8ms8JZWYY5f5S129ufTkRDPpJgemzCIb/5i6YgYXIjuzU6g4Y6DP2B9YnpHFK3n pLkgHDoNxwghh8HboBud0qinn1J1Td3V9LFuk+nK57EpXjh9V0t6MVMQPaToYuPmuCRj NMs9n/GZ5UojL7qzZD8HPx4b8fSqqsLdiTk2rER82PiB2qf3gdsaNnvKy34cdaeop6XM 19Xw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=DXXQnvZp; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b17si1517992eje.306.2020.08.24.11.36.45; Mon, 24 Aug 2020 11:36:45 -0700 (PDT) Received-SPF: pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=DXXQnvZp; spf=pass (google.com: domain of stable-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725946AbgHXSgm (ORCPT + 15 others); Mon, 24 Aug 2020 14:36:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44224 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726964AbgHXSga (ORCPT ); Mon, 24 Aug 2020 14:36:30 -0400 Received: from mail-pg1-x544.google.com (mail-pg1-x544.google.com [IPv6:2607:f8b0:4864:20::544]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BC465C061573; Mon, 24 Aug 2020 11:36:29 -0700 (PDT) Received: by mail-pg1-x544.google.com with SMTP id d19so4970625pgl.10; Mon, 24 Aug 2020 11:36:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3q7TgN3lxewyLqAIBfP9LryeeItLVKAQXHJvLwyIaWM=; b=DXXQnvZpQdBxWWuTfbWVY5FLOwytb0aSgYz9mPH9jwhVJ/ogBB2JP13Gr4jK7HM73T pc/bkX58u16w83O633eIk/waD+bT8E/aoCcgOOg01+1vouzqLJH0+zGQjpzQJFyg0FZM mauzJky23s4DKNw2+aNYSjn1sV7zZZvHGKo4/0BHmr5FKsNRLRZXrAxTyrW6tvy3xhDC /27GPtZ1Tm5+yzrDJZcaCnxIN0oRlJybvVM6DWCbdS2+r7BgXNLnHHCtzQ1epJHThmTq nuiTZWKKuwFHf58c5DAoL1PXrjj4dXftqHFIRq8jatyM8kSLLpJJ4ZP+WD0NdXeYZWYa 7m1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3q7TgN3lxewyLqAIBfP9LryeeItLVKAQXHJvLwyIaWM=; b=H46rtuU/8qmPWTXcVQgZaWHLWvju3V+uO7LRrDpWpdtshqKDIvicPTkb77+D076M1m hGbIXRm7qyO7Hqvn1u04AWH2dCweMhU13YKPUckktuTCiDjhIW2CUVuTVUlBdi5QZOWI qv3w0RXFQMWBJ4cNdm7Hv7paSJw9z6QhWQLcfzohmzppzHV/ffzO8UA33DPwi7PSXsYf k6um9vGekupd09EG8GOYBwFfpneubA1M2DDhRIxeZ30vXamH32nHNEZF20G5UUFKWl/h neTO80IBbYy0oe3lG8gvWKR85LX7Fokw4W1R7/les18JRQQ23lDbuNMvJbvlz9KQgBXv 09fA== X-Gm-Message-State: AOAM532pM1W/GYEZwroIsB0OBqQI82NKtiTxMoFfsTCMdVKagJtQewFW ubLo1fFeDvn2lirIAE1KfW8= X-Received: by 2002:a17:902:8690:: with SMTP id g16mr2883173plo.257.1598294189292; Mon, 24 Aug 2020 11:36:29 -0700 (PDT) Received: from stbirv-lnx-3.igp.broadcom.net ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id u191sm10235065pgu.56.2020.08.24.11.36.27 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 24 Aug 2020 11:36:28 -0700 (PDT) From: Florian Fainelli To: linux-arm-kernel@lists.infradead.org Cc: Will Deacon , Florian Fainelli , Greg KH , Will Deacon , stable@vger.kernel.org, Catalin Marinas , Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , Mark Rutland , Mark Brown , Kristina Martsenko , Sami Tolvanen , Andrew Jones , Ard Biesheuvel , Nick Desaulniers , Fangrui Song , open list , "open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)" Subject: [PATCH stable 4.9 v3 2/2] arm64: entry: Place an SB sequence following an ERET instruction Date: Mon, 24 Aug 2020 11:36:22 -0700 Message-Id: <1598294182-34012-3-git-send-email-f.fainelli@gmail.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1598294182-34012-1-git-send-email-f.fainelli@gmail.com> References: <1598294182-34012-1-git-send-email-f.fainelli@gmail.com> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Will Deacon commit 679db70801da9fda91d26caf13bf5b5ccc74e8e8 upstream Some CPUs can speculate past an ERET instruction and potentially perform speculative accesses to memory before processing the exception return. Since the register state is often controlled by a lower privilege level at the point of an ERET, this could potentially be used as part of a side-channel attack. This patch emits an SB sequence after each ERET so that speculation is held up on exception return. Signed-off-by: Will Deacon [florian: Adjust hyp-entry.S to account for the label added change to hyp/entry.S] Signed-off-by: Florian Fainelli --- arch/arm64/kernel/entry.S | 2 ++ arch/arm64/kvm/hyp/entry.S | 2 ++ arch/arm64/kvm/hyp/hyp-entry.S | 4 ++++ 3 files changed, 8 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index ca978d7d98eb..3408c782702c 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -255,6 +255,7 @@ alternative_insn eret, nop, ARM64_UNMAP_KERNEL_AT_EL0 .else eret .endif + sb .endm .macro get_thread_info, rd @@ -945,6 +946,7 @@ __ni_sys_trace: mrs x30, far_el1 .endif eret + sb .endm .align 11 diff --git a/arch/arm64/kvm/hyp/entry.S b/arch/arm64/kvm/hyp/entry.S index a360ac6e89e9..93704e6894d2 100644 --- a/arch/arm64/kvm/hyp/entry.S +++ b/arch/arm64/kvm/hyp/entry.S @@ -83,6 +83,7 @@ ENTRY(__guest_enter) // Do not touch any register after this! eret + sb ENDPROC(__guest_enter) ENTRY(__guest_exit) @@ -195,4 +196,5 @@ alternative_endif ldp x0, x1, [sp], #16 eret + sb ENDPROC(__fpsimd_guest_restore) diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index bf4988f9dae8..3675e7f0ab72 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -97,6 +97,7 @@ el1_sync: // Guest trapped into EL2 do_el2_call 2: eret + sb el1_hvc_guest: /* @@ -147,6 +148,7 @@ wa_epilogue: mov x0, xzr add sp, sp, #16 eret + sb el1_trap: get_vcpu_ptr x1, x0 @@ -198,6 +200,7 @@ el2_error: b.ne __hyp_panic mov x0, #(1 << ARM_EXIT_WITH_SERROR_BIT) eret + sb ENTRY(__hyp_do_panic) mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ @@ -206,6 +209,7 @@ ENTRY(__hyp_do_panic) ldr lr, =panic msr elr_el2, lr eret + sb ENDPROC(__hyp_do_panic) ENTRY(__hyp_panic)