From patchwork Tue Jan 16 14:23:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 124717 Delivered-To: patch@linaro.org Received: by 10.46.64.148 with SMTP id r20csp1031470lje; Tue, 16 Jan 2018 06:26:04 -0800 (PST) X-Google-Smtp-Source: ACJfBot6aCLe3GJKqscdZNfPUFZfP0sohpIpq3ayL1mTa0gt0QF6nA3cu6rHWAZyTQSmGtrhBevN X-Received: by 10.107.8.17 with SMTP id 17mr1578304ioi.35.1516112764162; Tue, 16 Jan 2018 06:26:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516112764; cv=none; d=google.com; s=arc-20160816; b=c1MTnZDtQTcax1o7MouTa2JCUCZRxCz/B4tYydgl97NNe08BcHJYN1r1cBstxyxDp5 BLDsrovEvJDEF16rONnrcE3J9M3G0fC3AZdxNy3etlRu7acsv7Q20maSKPaWgIxrnzBo IFWotCjQrsJZYZafAOGLkYI2qWZ9P9xWIaWqN2G/qhgFDV/gHWxbYAqksyyInbQrgX5/ 7lyS5KomqSsK+l/Cv0M9Dd1R2yohHncS+QYG/r/HLlp6s8tuuB/hrcni6tev4Og6Ifpk JTlIlkVfUSlwQRW/QhjDW/97NNqmVt2HWrTgLjOlIgm7ZvdOWSrCm66UK9mhJ7v9/obj zL0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:cc:references:in-reply-to:message-id:date:to :from:dkim-signature:arc-authentication-results; bh=4TL5uw22Umf0n76CVDgCgegipW2spMQeb/rngZ+j7Go=; b=w/OWL35sODRHyCKNlfbFhB+F7F7mOAk1BPMcezMeHIf1mYAtKnH/IQUzyYxZ0SDvzk 8NugpasNj22mgQxH27ukVl++SYvEsnYxGb+Y70lfEoWhkPUtIaa6cbe0N7Ci3yek+8ch QtDRlcq77LAKDJ8phv9QwM3O7kcSK8/6JLAF05V229GoahzdCGOvip1ehfsKPqYzB3xR 4kXvOE61zhKtkj8kSxt39BvTSrqIlXmb+Hq7xnnO6yvMuWmfgb8TxMh7pBnNQZtGlCaU e9qJItyMNghWRFmOflzBSvB5upA1VYVu6shNRRjAJCrfCfRrH1gxUEukTd0qZ6GPOxvi i1Ng== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=kzN+KlZc; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id a1si2394779itf.141.2018.01.16.06.26.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:26:04 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=kzN+KlZc; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9R-0002Ai-Lg; Tue, 16 Jan 2018 14:23:45 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9Q-0002AT-FJ for xen-devel@lists.xen.org; Tue, 16 Jan 2018 14:23:44 +0000 X-Inumbo-ID: a56e7d7f-fac8-11e7-b4a6-bc764e045a96 Received: from mail-wr0-x243.google.com (unknown [2a00:1450:400c:c0c::243]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id a56e7d7f-fac8-11e7-b4a6-bc764e045a96; Tue, 16 Jan 2018 15:22:14 +0100 (CET) Received: by mail-wr0-x243.google.com with SMTP id e41so15020080wre.9 for ; Tue, 16 Jan 2018 06:23:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=JNMJr7RATyQdr5qU1Sn2hNxlI8qztQWJpAZ36w1NOls=; b=kzN+KlZcfTmZRs7nUM53CH4M0aaX7JS8hp/81ZYSZu1WZroFdS70ubcFLoLRy/NKgg SWZsapcIjN0crzN1u8LvuGRsu7b+3kRoVeqPd6eg5u9XU+UoNZ5RT9MScPryNccgIgk6 f3Xu2EKp8CpWiDG4PxIWgHP8O+G8mtvcaY6ec= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=JNMJr7RATyQdr5qU1Sn2hNxlI8qztQWJpAZ36w1NOls=; b=JZqQPmHB3LxQJMUuAJoyDgXpALioKhouX88CxBY7W6aIpdIbU/lbiJjDrG3bsVNKTk 03IRts9ypGsDIJFZ6zdAhpvZ9M28VQ5HcY3Ej1PTFhNmORdq0uu8ha0tFUJmyoII7KWg 1hrQPox39MRI09lAO0tirUsdoS7RaKaz+J/hZrjpV3iV4txoXz+hSzqxPEE4bwCP5QY6 RANoFVedd4Ee0QlEbKUPvv+KAtmCG8Db3u6On57wvrd1+FnQDLSN4U4jhtEJgfyFPdP2 uiD8klqyVofiCyn4f43xdG2wjQzZeu+CQnuZAWOI180f/3j9gjpzFZ5i5M2XxE6r4aig 5rvg== X-Gm-Message-State: AKGB3mLMjrHiq1JpY9l2inmQ3NtQN+TrUcPV9Y2W6u9eqXprv/F98Ojm E8cItAId9HfjqoAiRRQ+woCNdWjW6kE= X-Received: by 10.223.151.20 with SMTP id r20mr24952070wrb.24.1516112622191; Tue, 16 Jan 2018 06:23:42 -0800 (PST) Received: from e108454-lin.cambridge.arm.com ([2001:41d0:1:6c23::1]) by smtp.gmail.com with ESMTPSA id m201sm1686886wma.13.2018.01.16.06.23.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:23:41 -0800 (PST) From: Julien Grall To: xen-devel@lists.xen.org Date: Tue, 16 Jan 2018 14:23:33 +0000 Message-Id: <20180116142337.24942-2-julien.grall@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180116142337.24942-1-julien.grall@linaro.org> References: <20180116142337.24942-1-julien.grall@linaro.org> Cc: sstabellini@kernel.org, Julien Grall , andre.przywara@linaro.org Subject: [Xen-devel] [PATCH 1/5] xen/arm: Introduce enable callback to enable a capabilities on each online CPU X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Once Xen knows what features/workarounds present on the platform, it might be necessary to configure each online CPU. Introduce a new callback "enable" that will be called on each online CPU to configure the "capability". The code is based on Linux v4.14 (where cpufeature.c comes from), the explanation of why using stop_machine_run is kept as we have similar problem in the future. Lastly introduce enable_errata_workaround that will be called once CPUs have booted and before the hardware domain is created. This is part of XSA-254. Signed-of-by: Julien Grall Reviewed-by: Stefano Stabellini --- xen/arch/arm/cpuerrata.c | 6 ++++++ xen/arch/arm/cpufeature.c | 29 +++++++++++++++++++++++++++++ xen/arch/arm/setup.c | 1 + xen/include/asm-arm/cpuerrata.h | 1 + xen/include/asm-arm/cpufeature.h | 3 +++ 5 files changed, 40 insertions(+) diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c index fe9e9facbe..772587c05a 100644 --- a/xen/arch/arm/cpuerrata.c +++ b/xen/arch/arm/cpuerrata.c @@ -64,6 +64,12 @@ void check_local_cpu_errata(void) { update_cpu_capabilities(arm_errata, "enabled workaround for"); } + +void __init enable_errata_workarounds(void) +{ + enable_cpu_capabilities(arm_errata); +} + /* * Local variables: * mode: C diff --git a/xen/arch/arm/cpufeature.c b/xen/arch/arm/cpufeature.c index 479c9fb011..525b45e22f 100644 --- a/xen/arch/arm/cpufeature.c +++ b/xen/arch/arm/cpufeature.c @@ -19,6 +19,7 @@ #include #include #include +#include #include DECLARE_BITMAP(cpu_hwcaps, ARM_NCAPS); @@ -40,6 +41,34 @@ void update_cpu_capabilities(const struct arm_cpu_capabilities *caps, } /* + * Run through the enabled capabilities and enable() it on all active + * CPUs. + */ +void __init enable_cpu_capabilities(const struct arm_cpu_capabilities *caps) +{ + for ( ; caps->matches; caps++ ) + { + if ( !cpus_have_cap(caps->capability) ) + continue; + + if ( caps->enable ) + { + int ret; + + /* + * Use stop_machine_run() as it schedules the work allowing + * us to modify PSTATE, instead of on_each_cpu() which uses + * an IPI, giving us a PSTATE that disappears when we + * return. + */ + ret = stop_machine_run(caps->enable, (void *)caps, NR_CPUS); + /* stop_machine_run should never fail at this stage of the boot. */ + BUG_ON(ret); + } + } +} + +/* * Local variables: * mode: C * c-file-style: "BSD" diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c index 16a3b1be8e..032a6a882d 100644 --- a/xen/arch/arm/setup.c +++ b/xen/arch/arm/setup.c @@ -849,6 +849,7 @@ void __init start_xen(unsigned long boot_phys_offset, * stop_machine (tasklets initialized via an initcall). */ apply_alternatives_all(); + enable_errata_workarounds(); /* Create initial domain 0. */ /* The vGIC for DOM0 is exactly emulating the hardware GIC */ diff --git a/xen/include/asm-arm/cpuerrata.h b/xen/include/asm-arm/cpuerrata.h index 8b158429c7..7de68361ff 100644 --- a/xen/include/asm-arm/cpuerrata.h +++ b/xen/include/asm-arm/cpuerrata.h @@ -5,6 +5,7 @@ #include void check_local_cpu_errata(void); +void enable_errata_workarounds(void); #ifdef CONFIG_HAS_ALTERNATIVE diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h index f00b6dbd39..21c65e198c 100644 --- a/xen/include/asm-arm/cpufeature.h +++ b/xen/include/asm-arm/cpufeature.h @@ -74,6 +74,7 @@ struct arm_cpu_capabilities { const char *desc; u16 capability; bool (*matches)(const struct arm_cpu_capabilities *); + int (*enable)(void *); /* Called on every active CPUs */ union { struct { /* To be used for eratum handling only */ u32 midr_model; @@ -85,6 +86,8 @@ struct arm_cpu_capabilities { void update_cpu_capabilities(const struct arm_cpu_capabilities *caps, const char *info); +void enable_cpu_capabilities(const struct arm_cpu_capabilities *caps); + #endif /* __ASSEMBLY__ */ #endif From patchwork Tue Jan 16 14:23:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 124720 Delivered-To: patch@linaro.org Received: by 10.46.64.148 with SMTP id r20csp1031488lje; Tue, 16 Jan 2018 06:26:07 -0800 (PST) X-Google-Smtp-Source: ACJfBovfHnfmGqK2WBbOuIOnagG+e+St+rGlY3v7CQd1Fin7aB73WS4klN0qtuS4EvFg5QwgajUS X-Received: by 10.107.232.7 with SMTP id f7mr39075471ioh.171.1516112767875; Tue, 16 Jan 2018 06:26:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516112767; cv=none; d=google.com; s=arc-20160816; b=XEmWYUW3QK1wpL4ewGSiGufSo5Km2o2ylLs0uSvmUA2L0WAXRK0nlpWO/yS2CAO2V3 bDVul+IetupiCmaSepAO1Cd6jk5sOL3XIqEJPlXiagm6dDXuzdLBU+2xKoHyb8dzmCLO 3FCVvUE9f5Lq6MwyvzM8cbApFXPua/PFGwXF0+IUtBXwqImMPbRYV+eJ5NITOepP/YvW KHJ3YxXfBGAWlYg115ESMwr4sQS1z3BxoXdtyvb5YXxhdZflskNUO+4sHEWlGT0M77/i eBXQRXrIF7tZ36gyV1UfZwS9n7QHvCrTqstEHbAYBJOnjOGYHEq3fGgjIwbUnZ7MSgNz JDQw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:cc:references:in-reply-to:message-id:date:to :from:dkim-signature:arc-authentication-results; bh=n2dsYas2ZFQnwz91NdaHkLMRMnBbKgvoy2XsPeJb2h0=; b=MpJ1QBpNKte8bvFNpGJe4NPrOHiIfI+Hmfy4MN/O2qw2mFBzzbHDSOCR3LtylJqWvv YWN7DtluGiqsmlXFqIeB5XgCCQdnUGXp4slD0LweVdies+Ici79OQwiU1DTaiYsVpMld 895mhwU1nGEHUIxogh3b/MN7+HByNX5j8SVq4FQmU+z2rEHwaxXFowtQ9nM6JezVJfBu vvY1+fzPCyMRgv/rWa5bAEVbpjgSlwYLhONLOApf0LzlDAeLwY7U6eqWueGsTsvQPlhn UbzWRsCGOafXxSwbAfcrkLCKiVL2EPgH47NvhgEy97m/Wi/Qz61E/Qo5xUhaUB9PoSmF It2w== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=jj1/jeDq; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id n64si2079739ioe.147.2018.01.16.06.26.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:26:07 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=jj1/jeDq; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9T-0002Ce-6S; Tue, 16 Jan 2018 14:23:47 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9R-0002Ah-OJ for xen-devel@lists.xen.org; Tue, 16 Jan 2018 14:23:45 +0000 X-Inumbo-ID: a607d516-fac8-11e7-b4a6-bc764e045a96 Received: from mail-wm0-x241.google.com (unknown [2a00:1450:400c:c09::241]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id a607d516-fac8-11e7-b4a6-bc764e045a96; Tue, 16 Jan 2018 15:22:15 +0100 (CET) Received: by mail-wm0-x241.google.com with SMTP id b141so8807000wme.1 for ; Tue, 16 Jan 2018 06:23:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=8wxFB9wGv1K4MQacAPmJ6ud6e+OKgjR/JREbUeMSJAA=; b=jj1/jeDqywapzlSOUFiM2E05dhkqDXZNoExbgtJ/u+M7RG9WaYBLwSmgsOwZwamtgl FnNht0BF/fLNNHaTwET60GRcd09CQIle4eI7ZcPWcKfrFon4nNqw+YjIphlqOTnL9EUG sAJYrJT/qPP66pNB1IvkZgtxEu1cZiRxHe1Z8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=8wxFB9wGv1K4MQacAPmJ6ud6e+OKgjR/JREbUeMSJAA=; b=YsiJNcydFNxNiFCZua20e5z9FjQ4drTG6oRzBMqGaj4G/BGnvb3kudKSpE6QP+VtK3 ge+UGOuGVBamEHB6nZMMuhuEWj6FSmphVn3VwKsIYyAOvyzs6TCVfIdqQuOZNcW1jKxe r1br2OJloV8vWVPv+XMwbWBJDsWKXAsKiDNqG977NXG7OGeXeIq77fAFwMaQiclVasVj ympJTy7UUz6FtLhMrUvpiLpGLTNA81yWVZynNMplKz22AptTiQDxf6vu+zARgYJL/3Fk juakuXNVWvBmP+GlI6V6Rf7bgIohftph38js9GRatLppbQ5RFOsjVnEqderxXSTkNYFm +wYw== X-Gm-Message-State: AKwxytfWL/KMnR++2ld0HRLj4+g00I23gfw7WeM1z2B2l2nTwFKggiMb yH4/MMwKSPTaCmKI8nquqEZr5N0fY50= X-Received: by 10.28.237.20 with SMTP id l20mr12491731wmh.15.1516112623231; Tue, 16 Jan 2018 06:23:43 -0800 (PST) Received: from e108454-lin.cambridge.arm.com ([2001:41d0:1:6c23::1]) by smtp.gmail.com with ESMTPSA id m201sm1686886wma.13.2018.01.16.06.23.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:23:42 -0800 (PST) From: Julien Grall To: xen-devel@lists.xen.org Date: Tue, 16 Jan 2018 14:23:34 +0000 Message-Id: <20180116142337.24942-3-julien.grall@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180116142337.24942-1-julien.grall@linaro.org> References: <20180116142337.24942-1-julien.grall@linaro.org> Cc: sstabellini@kernel.org, Julien Grall , andre.przywara@linaro.org Subject: [Xen-devel] [PATCH 2/5] xen/arm64: Add missing MIDR values for Cortex-A72, A73 and A75 X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Cortex-A72, A73 and A75 MIDR will be used to a follow-up for hardening the branch predictor. This is part of XSA-254. Signed-off-by: Julien Grall Acked-by: Stefano Stabellini --- xen/include/asm-arm/processor.h | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h index 65eb1071e1..3edab1b893 100644 --- a/xen/include/asm-arm/processor.h +++ b/xen/include/asm-arm/processor.h @@ -47,10 +47,16 @@ #define ARM_CPU_PART_CORTEX_A15 0xC0F #define ARM_CPU_PART_CORTEX_A53 0xD03 #define ARM_CPU_PART_CORTEX_A57 0xD07 +#define ARM_CPU_PART_CORTEX_A72 0xD08 +#define ARM_CPU_PART_CORTEX_A73 0xD09 +#define ARM_CPU_PART_CORTEX_A75 0xD0A #define MIDR_CORTEX_A15 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A15) #define MIDR_CORTEX_A53 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A53) #define MIDR_CORTEX_A57 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A57) +#define MIDR_CORTEX_A72 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A72) +#define MIDR_CORTEX_A73 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A73) +#define MIDR_CORTEX_A75 MIDR_CPU_MODEL(ARM_CPU_IMP_ARM, ARM_CPU_PART_CORTEX_A75) /* MPIDR Multiprocessor Affinity Register */ #define _MPIDR_UP (30) From patchwork Tue Jan 16 14:23:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 124718 Delivered-To: patch@linaro.org Received: by 10.46.64.148 with SMTP id r20csp1031474lje; Tue, 16 Jan 2018 06:26:05 -0800 (PST) X-Google-Smtp-Source: ACJfBosZCccj5FDzantMFxr2a/Jfo3UvjKimME9Qx3q6jK/BGNYXN2qNPYPgOvjeY44ki6ksKTcQ X-Received: by 10.107.24.195 with SMTP id 186mr35897539ioy.185.1516112765262; Tue, 16 Jan 2018 06:26:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516112765; cv=none; d=google.com; s=arc-20160816; b=OzHgO9uCbkuutfDDMec8nbMtnl38wO2/gAWS5Hfoztue0pGej9u3cx7BqZVxJFY/RO 6pmr8Jnl4S4HgRH58aoDqNs1C3L26yghlvhZzatrmLgcarwqBfPahacPh/zKlyvYcNvx RgjWoXA+E55zikMxHadktOWYAg93YUUj7eI4oLPCH1q2bGgnvefMjaKyqYHjOaerpfd9 9wok8ZDBxnlMqfl9ki0VRBMl9WHdJ+emI9lCDcpV+uNzfPQP/uFhsd1AO+rEC4M183gF Qm/WWPa3QHVTICRzjeUaJJsNeZcAv17kqedH/YjCgMaDy4fPaPW2l7bPK2n8ndbUItxN fkmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:cc:references:in-reply-to:message-id:date:to :from:dkim-signature:arc-authentication-results; bh=YNnZj+bD1DNNAIeQ4ktbCz/j0ducO3XCd9Ir+M+fMWg=; b=kcGIlvdJctLRdEfZ7xtY8HE32vZbXDNzuAJAUsKRdh6XT0WMmSfwDBoNIZ2F2/Ghus jQmcWRTcXHF/Q9bynyimNuO2uMaTqPIrezYNgEbOl07AsNzfad0UK0bLSnRrqDd8u3QX dtKq5mPnOvtez4lYCKuOpQ3QspSYHG21D8X3qhSBHf6YbuamnUDy6BNXWrKqvDLM6kPR tktrZOVqT1VPNp0xbgku2NvNjdHKclSfreRBDmeGhdJKn8vodGIBVY7LoiYM9EqgZyPt R4jEjjvn38N8YfJ/E7fMishIyp4QMG/brJEpY3hOC+v2bPdDsECQzevML+DktlZxSRfO qwMA== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=ZVjVpaZ4; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id p36si2398563ioo.210.2018.01.16.06.26.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:26:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=ZVjVpaZ4; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9T-0002D7-DR; Tue, 16 Jan 2018 14:23:47 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9S-0002Bt-E3 for xen-devel@lists.xen.org; Tue, 16 Jan 2018 14:23:46 +0000 X-Inumbo-ID: a6c10863-fac8-11e7-b4a6-bc764e045a96 Received: from mail-wr0-x242.google.com (unknown [2a00:1450:400c:c0c::242]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id a6c10863-fac8-11e7-b4a6-bc764e045a96; Tue, 16 Jan 2018 15:22:16 +0100 (CET) Received: by mail-wr0-x242.google.com with SMTP id p6so15392032wrd.0 for ; Tue, 16 Jan 2018 06:23:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=aaPbjjfF4h4zTTt0bbZObyPXY/NzJIAgx43hpdCABw8=; b=ZVjVpaZ4iYVNGwohGdaJbp0ibXt9Hk57NZz5cB6NzCZutzQHtcc0FCxFO8PlFsm+Xp YJ8rJWCL/BDdyMD0TRmc34Vv3f6QGrnn/lWdadH1a7e8rXhloeYtxroqW1/Hwo8HcrO8 l8MbUSWZwoghhJKim0+k2dyLI630yD6RKa8EA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=aaPbjjfF4h4zTTt0bbZObyPXY/NzJIAgx43hpdCABw8=; b=DbnON4c/v8071Yt6bju4iIzDz+KoAwyrYy0leogkn7U84e0Ct0OX0VnYjioMnDHCnZ 2ugFPg/XBTcFyKSX+fXQ3PcBbA2oZ5Oq7fTN36a/g3ernbQaxoTn/DvRIaTJcRYclGuj KhlRH/doEI3UhtjWUq4NKHEfRPdN58JllXT2gdhENPXEDxp8Dy8cAH3lty4Sigpjd/qI RBxrCwAKO7gBmJU+9jd49qIJR1UOsdKvH8sBfKv88DMdXA36O7kaoM0uYK8w7gML7z7j 7Gu9uo4EYH6CaOQWzoVHtoLej2m0AgjeL8qRmP/9TC9ChmttKQhAn6NOch0DGN1DO4ZY KiaQ== X-Gm-Message-State: AKwxytfpMTreSemeqZgSgH5mEfgOdFCCjI5tu2xjyYhrl5bu9lUsYkiK kR1+xOexg33gFTYWTpKAs8CTGTwoRdU= X-Received: by 10.223.172.6 with SMTP id v6mr3032714wrc.269.1516112624439; Tue, 16 Jan 2018 06:23:44 -0800 (PST) Received: from e108454-lin.cambridge.arm.com ([2001:41d0:1:6c23::1]) by smtp.gmail.com with ESMTPSA id m201sm1686886wma.13.2018.01.16.06.23.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:23:43 -0800 (PST) From: Julien Grall To: xen-devel@lists.xen.org Date: Tue, 16 Jan 2018 14:23:35 +0000 Message-Id: <20180116142337.24942-4-julien.grall@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180116142337.24942-1-julien.grall@linaro.org> References: <20180116142337.24942-1-julien.grall@linaro.org> Cc: sstabellini@kernel.org, Julien Grall , andre.przywara@linaro.org Subject: [Xen-devel] [PATCH 3/5] xen/arm: cpuerrata: Add MIDR_ALL_VERSIONS X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Introduce a new macro MIDR_ALL_VERSIONS to match all variant/revision of a given CPU model. This is part of XSA-254. Signed-off-by: Julien Grall Reviewed-by: Stefano Stabellini --- xen/arch/arm/cpuerrata.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c index 772587c05a..c50d3331f2 100644 --- a/xen/arch/arm/cpuerrata.c +++ b/xen/arch/arm/cpuerrata.c @@ -7,6 +7,12 @@ .midr_range_min = min, \ .midr_range_max = max +#define MIDR_ALL_VERSIONS(model) \ + .matches = is_affected_midr_range, \ + .midr_model = model, \ + .midr_range_min = 0, \ + .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK) + static bool __maybe_unused is_affected_midr_range(const struct arm_cpu_capabilities *entry) { From patchwork Tue Jan 16 14:23:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 124722 Delivered-To: patch@linaro.org Received: by 10.46.64.148 with SMTP id r20csp1031814lje; Tue, 16 Jan 2018 06:27:20 -0800 (PST) X-Google-Smtp-Source: ACJfBostz92ko6vJwJI6/hImTSsyxbCT8K21oK6z+EQ/j9ETmjiBh/byUKzqdnPz2nn0c0G4xrXf X-Received: by 10.107.175.103 with SMTP id y100mr12404074ioe.45.1516112840063; Tue, 16 Jan 2018 06:27:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516112840; cv=none; d=google.com; s=arc-20160816; b=y0XYJmgFk0o+huop1HZKi2MGrC7OM/7ZZlOeSFN9ikGShO2UJSOWHAXna0c0csFHUm 6RCJrfn26LtNQgQ3zMz5yOnICE1ZDbP/m2uh01Ce5WteCeu9jrTRAunOWbqyTTPltyfI Qck252+28P2jhEqtGcetKyzLOW5dZkTZw7RUbH/DEpo9lF13FVhfMaHCDVDRachnyPQm A33DsDP3Pm812d8EtKlHqUh8wbA+o1b4wk4fE1ZPNmS2w7gMsT8rV8seJPor6BBHYNHT idHvKsR8MpHUdcyiR/+W4fFJFVpjAUhdpne1bTkZjNFhBBKyavM3MQa+rX4QcBQSFeUN M+Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:cc:references:in-reply-to:message-id:date:to :from:dkim-signature:arc-authentication-results; bh=C+Sq0XqTXZF2IUG0+3YW9rWJWIYOVHuOhXjz8E6t38o=; b=nIIL+demrjninmC2Af9mbqGOi+IwYHExtYI2sxZoCpcS19khtRGr9sbxlKFTQwK8Qb WW1TLPx3Gz6bZhimsKFSUVIYuJ27reXvWsI8Ma7fl7t+OPFgphM/YBYPoTWjsaQTfPX8 AMBKtxuHq3ZTEDOlQNxY3bky4JX12n79mhBOnBo3zTvKHMfC3SD/bo4mbxUuD89lYp+4 BgwRM8PMSerbd7q8M6KgldtcJj89TuhYcO8JJtifAjZPn1VVOeVpUKhw/g3GPf+asAAG M7aHG/Yqx5hcEQgduEYFNpiM2B5gr9EsL9pnuJCse8trpdziBLiSJ74A3e+GWPP/oQHC bsxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=AdHtaXRA; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id c66si2020064iof.274.2018.01.16.06.27.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:27:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=AdHtaXRA; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebSAY-0002aD-V0; Tue, 16 Jan 2018 14:24:54 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebSAX-0002Zh-Jh for xen-devel@lists.xen.org; Tue, 16 Jan 2018 14:24:53 +0000 X-Inumbo-ID: 58f6ff7d-fac9-11e7-b0d7-9f685aff125f Received: from mail-wm0-f68.google.com (unknown [74.125.82.68]) by us1-amaz-eas1.inumbo.com (Halon) with ESMTPS id 58f6ff7d-fac9-11e7-b0d7-9f685aff125f; Tue, 16 Jan 2018 14:27:15 +0000 (UTC) Received: by mail-wm0-f68.google.com with SMTP id i186so8694955wmi.4 for ; Tue, 16 Jan 2018 06:24:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0VlAQncoCqk/V/R+7mNHJ4xuEOvJNe5x26k1IPzTjh0=; b=AdHtaXRAdy1ub9+NPyXybnQTuRMrPc1AEm/4gc971I8Z70dpIt9KJ6fenb0TSSKzXP 6JKyoPmaC+HZmPXScXgNFnMooi2IAOC3t2vVEqwoaGW7t/VXfdRjp6Ws+zQGNgmqDsM0 P7r+z/xJqzs0psZKzMDUnzio4e7ONMy2rO4e4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0VlAQncoCqk/V/R+7mNHJ4xuEOvJNe5x26k1IPzTjh0=; b=fWhmiQ6scZ/fWfNP/NXkBOv6TiUZlj5UFRrrGOxO9YbB+YODEsh2lIhXXoY6Kics06 0DCRjKTCTMz7S+wIZxznFfL42ovFeMuHIut+XrT/DZwN/inqXi5oOR599+ufshINJooN mvYT3jWCZdFgsLFPAMTqbd+2RtD+EBvjxwTSFf3Z+Z92nwX3o/FkrOBbWcZ1zJDHQjRT SuPJa7YaBqiLl8s2wgRlNrBh+d7rvb3eGBQchEDxcZZM3Iy/7SE6nMYGsR6IvGvszz9v T7mCDy04t8mVC+80XvrZ7dhLzIlZ9YNG02QnXvc6DKp7TiXg2+3omR8SYvACArIZLBeW O1dw== X-Gm-Message-State: AKwxytecv9g30mbYCxLIW2gOASC2P/umxyD8t9j5QS/6+4E0hWZ9pctZ HBok8vnle8XIxM3CMPm4U9LZ+JbQx0Y= X-Received: by 10.28.156.67 with SMTP id f64mr14059728wme.11.1516112625577; Tue, 16 Jan 2018 06:23:45 -0800 (PST) Received: from e108454-lin.cambridge.arm.com ([2001:41d0:1:6c23::1]) by smtp.gmail.com with ESMTPSA id m201sm1686886wma.13.2018.01.16.06.23.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:23:45 -0800 (PST) From: Julien Grall To: xen-devel@lists.xen.org Date: Tue, 16 Jan 2018 14:23:36 +0000 Message-Id: <20180116142337.24942-5-julien.grall@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180116142337.24942-1-julien.grall@linaro.org> References: <20180116142337.24942-1-julien.grall@linaro.org> Cc: sstabellini@kernel.org, Julien Grall , andre.przywara@linaro.org Subject: [Xen-devel] [PATCH 4/5] xen/arm64: Add skeleton to harden the branch predictor aliasing attacks X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Aliasing attacked against CPU branch predictors can allow an attacker to redirect speculative control flow on some CPUs and potentially divulge information from one context to another. This patch adds initial skeleton code behind a new Kconfig option to enable implementation-specific mitigations against these attacks for CPUs that are affected. Most of the mitigations will have to be applied when entering to the hypervisor from the guest context. For safety, it is applied at every exception entry. So there are potential for optimizing when receiving an exception at the same level. Because the attack is against branch predictor, it is not possible to safely use branch instruction before the mitigation is applied. Therefore, this has to be done in the vector entry before jump to the helper handling a given exception. On Arm64, each vector can hold 32 instructions. This leave us 31 instructions for the mitigation. The last one is the branch instruction to the helper. Because a platform may have CPUs with different micro-architectures, per-CPU vector table needs to be provided. Realistically, only a few different mitigations will be necessary. So provide a small set of vector tables. They will be re-used and patch with the mitigations on-demand. This is based on the work done in Linux (see [1]). This is part of XSA-254. [1] git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git branch ktpi Signed-off-by: Julien Grall Reviewed-by: Stefano Stabellini --- xen/arch/arm/Kconfig | 20 ++++++ xen/arch/arm/arm64/Makefile | 1 + xen/arch/arm/arm64/bpi.S | 64 ++++++++++++++++++ xen/arch/arm/cpuerrata.c | 142 +++++++++++++++++++++++++++++++++++++++ xen/arch/arm/traps.c | 5 +- xen/include/asm-arm/cpuerrata.h | 1 + xen/include/asm-arm/cpufeature.h | 3 +- xen/include/asm-arm/processor.h | 5 +- 8 files changed, 237 insertions(+), 4 deletions(-) create mode 100644 xen/arch/arm/arm64/bpi.S diff --git a/xen/arch/arm/Kconfig b/xen/arch/arm/Kconfig index f58019d6ed..06fd85cc77 100644 --- a/xen/arch/arm/Kconfig +++ b/xen/arch/arm/Kconfig @@ -171,6 +171,26 @@ config ARM64_ERRATUM_834220 endmenu +config HARDEN_BRANCH_PREDICTOR + bool "Harden the branch predictor against aliasing attacks" if EXPERT + default y + help + Speculation attacks against some high-performance processors rely on + being able to manipulate the branch predictor for a victim context by + executing aliasing branches in the attacker context. Such attacks + can be partially mitigated against by clearing internal branch + predictor state and limiting the prediction logic in some situations. + + This config option will take CPU-specific actions to harden the + branch predictor against aliasing attacks and may rely on specific + instruction sequences or control bits being set by the system + firmware. + + If unsure, say Y. + +config ARM64_HARDEN_BRANCH_PREDICTOR + def_bool y if ARM_64 && HARDEN_BRANCH_PREDICTOR + source "common/Kconfig" source "drivers/Kconfig" diff --git a/xen/arch/arm/arm64/Makefile b/xen/arch/arm/arm64/Makefile index 718fe44455..bb5c610b2a 100644 --- a/xen/arch/arm/arm64/Makefile +++ b/xen/arch/arm/arm64/Makefile @@ -1,6 +1,7 @@ subdir-y += lib obj-y += cache.o +obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o obj-$(EARLY_PRINTK) += debug.o obj-y += domctl.o obj-y += domain.o diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S new file mode 100644 index 0000000000..6cc2f17529 --- /dev/null +++ b/xen/arch/arm/arm64/bpi.S @@ -0,0 +1,64 @@ +/* + * Contains CPU specific branch predictor invalidation sequences + * + * Copyright (C) 2018 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +.macro ventry target + .rept 31 + nop + .endr + b \target +.endm + +.macro vectors target + ventry \target + 0x000 + ventry \target + 0x080 + ventry \target + 0x100 + ventry \target + 0x180 + + ventry \target + 0x200 + ventry \target + 0x280 + ventry \target + 0x300 + ventry \target + 0x380 + + ventry \target + 0x400 + ventry \target + 0x480 + ventry \target + 0x500 + ventry \target + 0x580 + + ventry \target + 0x600 + ventry \target + 0x680 + ventry \target + 0x700 + ventry \target + 0x780 +.endm + +/* + * Populate 4 vector tables. This will cover up to 4 different + * micro-architectures in a system. + */ + .align 11 +ENTRY(__bp_harden_hyp_vecs_start) + .rept 4 + vectors hyp_traps_vector + .endr +ENTRY(__bp_harden_hyp_vecs_end) + +/* + * Local variables: + * mode: ASM + * indent-tabs-mode: nil + * End: + */ diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c index c50d3331f2..76d98e771d 100644 --- a/xen/arch/arm/cpuerrata.c +++ b/xen/arch/arm/cpuerrata.c @@ -1,6 +1,148 @@ +#include +#include +#include +#include +#include +#include #include #include +/* Override macros from asm/page.h to make them work with mfn_t */ +#undef virt_to_mfn +#define virt_to_mfn(va) _mfn(__virt_to_mfn(va)) + +/* Hardening Branch predictor code for Arm64 */ +#ifdef CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR + +#define VECTOR_TABLE_SIZE SZ_2K + +/* + * Number of available table vectors (this should be in-sync with + * arch/arm64/bpi.S + */ +#define NR_BPI_HYP_VECS 4 + +extern char __bp_harden_hyp_vecs_start[], __bp_harden_hyp_vecs_end[]; + +/* + * Key for each slot. This is used to find whether a specific workaround + * had a slot assigned. + * + * The key is virtual address of the vector workaround + */ +static uintptr_t bp_harden_slot_key[NR_BPI_HYP_VECS]; + +/* + * [hyp_vec_start, hyp_vec_end[ corresponds to the first 31 instructions + * of each vector. The last (i.e 32th) instruction is used to branch to + * the original entry. + * + * Those instructions will be copied on each vector to harden them. + */ +static bool copy_hyp_vect_bpi(unsigned int slot, const char *hyp_vec_start, + const char *hyp_vec_end) +{ + void *dst_remapped; + const void *dst = __bp_harden_hyp_vecs_start + slot * VECTOR_TABLE_SIZE; + unsigned int i; + mfn_t dst_mfn = virt_to_mfn(dst); + + BUG_ON(((hyp_vec_end - hyp_vec_start) / 4) > 31); + + /* + * Vectors are part of the text that are mapped read-only. So re-map + * the vector table to be able to update vectors. + */ + dst_remapped = __vmap(&dst_mfn, + 1UL << get_order_from_bytes(VECTOR_TABLE_SIZE), + 1, 1, PAGE_HYPERVISOR, VMAP_DEFAULT); + if ( !dst_remapped ) + return false; + + dst_remapped += (vaddr_t)dst & ~PAGE_MASK; + + for ( i = 0; i < VECTOR_TABLE_SIZE; i += 0x80 ) + { + memcpy(dst_remapped + i, hyp_vec_start, hyp_vec_end - hyp_vec_start); + } + + clean_dcache_va_range(dst_remapped, VECTOR_TABLE_SIZE); + invalidate_icache(); + + vunmap(dst_remapped); + + return true; +} + +static bool __maybe_unused +install_bp_hardening_vec(const struct arm_cpu_capabilities *entry, + const char *hyp_vec_start, + const char *hyp_vec_end) +{ + static int last_slot = -1; + static DEFINE_SPINLOCK(bp_lock); + unsigned int i, slot = -1; + bool ret = true; + + /* + * Enable callbacks are called on every CPU based on the + * capabilities. So double-check whether the CPU matches the + * entry. + */ + if ( !entry->matches(entry) ) + return true; + + /* + * No need to install hardened vector when the processor has + * ID_AA64PRF0_EL1.CSV2 set. + */ + if ( cpu_data[smp_processor_id()].pfr64.csv2 ) + return true; + + spin_lock(&bp_lock); + + /* + * Look up whether the hardening vector had a slot already + * assigned. + */ + for ( i = 0; i < 4; i++ ) + { + if ( bp_harden_slot_key[i] == (uintptr_t)hyp_vec_start ) + { + slot = i; + break; + } + } + + if ( slot == -1 ) + { + last_slot++; + /* Check we don't overrun the number of slots available. */ + BUG_ON(NR_BPI_HYP_VECS <= last_slot); + + slot = last_slot; + ret = copy_hyp_vect_bpi(slot, hyp_vec_start, hyp_vec_end); + + /* Only update the slot if the copy succeeded. */ + if ( ret ) + bp_harden_slot_key[slot] = (uintptr_t)hyp_vec_start; + } + + if ( ret ) + { + /* Install the new vector table. */ + WRITE_SYSREG((vaddr_t)(__bp_harden_hyp_vecs_start + slot * VECTOR_TABLE_SIZE), + VBAR_EL2); + isb(); + } + + spin_unlock(&bp_lock); + + return ret; +} + +#endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */ + #define MIDR_RANGE(model, min, max) \ .matches = is_affected_midr_range, \ .midr_model = model, \ diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c index 013c1600ec..a3e4919751 100644 --- a/xen/arch/arm/traps.c +++ b/xen/arch/arm/traps.c @@ -160,7 +160,10 @@ __initcall(update_serrors_cpu_caps); void init_traps(void) { - /* Setup Hyp vector base */ + /* + * Setup Hyp vector base. Note they might get updated with the + * branch predictor hardening. + */ WRITE_SYSREG((vaddr_t)hyp_traps_vector, VBAR_EL2); /* Trap Debug and Performance Monitor accesses */ diff --git a/xen/include/asm-arm/cpuerrata.h b/xen/include/asm-arm/cpuerrata.h index 7de68361ff..23ebf367ea 100644 --- a/xen/include/asm-arm/cpuerrata.h +++ b/xen/include/asm-arm/cpuerrata.h @@ -1,6 +1,7 @@ #ifndef __ARM_CPUERRATA_H__ #define __ARM_CPUERRATA_H__ +#include #include #include diff --git a/xen/include/asm-arm/cpufeature.h b/xen/include/asm-arm/cpufeature.h index 21c65e198c..e557a095af 100644 --- a/xen/include/asm-arm/cpufeature.h +++ b/xen/include/asm-arm/cpufeature.h @@ -42,8 +42,9 @@ #define LIVEPATCH_FEATURE 4 #define SKIP_SYNCHRONIZE_SERROR_ENTRY_EXIT 5 #define SKIP_CTXT_SWITCH_SERROR_SYNC 6 +#define ARM_HARDEN_BRANCH_PREDICTOR 7 -#define ARM_NCAPS 7 +#define ARM_NCAPS 8 #ifndef __ASSEMBLY__ diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h index 3edab1b893..466da5da86 100644 --- a/xen/include/asm-arm/processor.h +++ b/xen/include/asm-arm/processor.h @@ -385,8 +385,9 @@ struct cpuinfo_arm { unsigned long fp:4; /* Floating Point */ unsigned long simd:4; /* Advanced SIMD */ unsigned long gic:4; /* GIC support */ - unsigned long __res0:4; - unsigned long __res1; + unsigned long __res0:28; + unsigned long csv2:4; + unsigned long __res1:4; }; } pfr64; From patchwork Tue Jan 16 14:23:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 124721 Delivered-To: patch@linaro.org Received: by 10.46.64.148 with SMTP id r20csp1031524lje; Tue, 16 Jan 2018 06:26:16 -0800 (PST) X-Google-Smtp-Source: ACJfBosypnisZHvNiOWxzhhr8hsADJU6sLgskrF3ZiQ1G3lLIwgSoZCp4Lj8JAZ5McNh3tOr8PQi X-Received: by 10.36.189.134 with SMTP id x128mr18460302ite.39.1516112776040; Tue, 16 Jan 2018 06:26:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516112776; cv=none; d=google.com; s=arc-20160816; b=QEmOweQLZFNhbFJ7UmLgP/wt6gG+WXg8jg4ErSDCbkB07aUSw0OP5AqXNk08G/z7SG ipZsbpG9oj50Z+mzSKFrmDpczU/bSK59mNPiglNzbzuTrfcV9jkSn8yBKHM/qb6SXQZ3 kSYS5hezSkpDszc6Gj/HplFv6/Fe6JIB+pk0EbSwAAkNFWWEn4f2BZCixasfyRusabbi FXYgbDy3uw4agYWPUHgiETXnKyIQ/lpdJXN1n+AcjtBtRANVMfp6sMs6iQFbyoqx9ALr KYkiywzeBXN0N+u9KQ/n/TXsF5j3MBG5EnztxNufqtYf2EBT8n9rRy5LuPB5WDBZgt8p TPYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:cc:references:in-reply-to:message-id:date:to :from:dkim-signature:arc-authentication-results; bh=wxhHc7T/VS6IoNsNbTh5Ndd8AmRvvKHY2CFDNyErlS8=; b=vL0nedNTcjoEG3M51Msl/N/+8YMZa2U/gS7lRJlj6SI1ySp+gVGWsgFf5p5Qun/+25 0WbEPCKBmGJnZD89JkDswtQALH9p3BXG1otia6l8uQ6QLH5VoXTmIHTs6Y9tjjMlLv/u AIa4mjOdcq75AQMpPzSFLbu64Dng86aaf2fzKVfiy7BiyqFo9tDTIzGmiUg6bmXs8dCv weISCcFUehRRbmJ7eKysb+8S70CfkQrAi4LRwlKPlpTQ7MMb//yaO0iK2qyhRsWBsBZr XDasI9qwAzojiLXZWgP7CSgkPREysUlTtGC+4q5rYpvddB+DkVaXX6Ob+M8jDf7OFb4L QG3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=dxJXX421; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id m15si2085687itg.4.2018.01.16.06.26.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:26:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org header.s=google header.b=dxJXX421; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9W-0002Fa-Kx; Tue, 16 Jan 2018 14:23:50 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ebS9V-0002F5-OL for xen-devel@lists.xen.org; Tue, 16 Jan 2018 14:23:49 +0000 X-Inumbo-ID: a87072ec-fac8-11e7-b4a6-bc764e045a96 Received: from mail-wm0-x244.google.com (unknown [2a00:1450:400c:c09::244]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTPS id a87072ec-fac8-11e7-b4a6-bc764e045a96; Tue, 16 Jan 2018 15:22:19 +0100 (CET) Received: by mail-wm0-x244.google.com with SMTP id 143so8677520wma.5 for ; Tue, 16 Jan 2018 06:23:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=z38FWZGwLpwaZknoKzEIsRzrKjjg3ec9C51vK9E0aak=; b=dxJXX4211Mg7qq7GkY7SalAr3aHpT8VDRWeZOeaRvTyq2TSpVtbH78rgX6Kd9N5d58 H7oWlzUPc7PlGI0JJQ2Q5I4LcuX5WZ5r8wjBJORJ1nAYuTXZMVuUV6qK0VSDheA2r2y9 ++0Ujpw8AhiiaewXLyTwiqnKBXzMUMKj8VI4Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=z38FWZGwLpwaZknoKzEIsRzrKjjg3ec9C51vK9E0aak=; b=TzuGhL/LV/zljl3MCoj16YY16znk1+eRN5VVz2FDMTAD3YPpVxssT5KT+Hw5wyyqEq KpPzoS8izfd41YQZZ5qUZ1rQQ5URqdPOzFC1cGAnxEYCPI8N2nc1GTXqgMQn4Gq+01rT qaVUaueK81GeEBDfFF4VT3INhRZxiIhrAu3UdECdoBNR9jd+G0oFftledt1Ms4i/3f++ 6wxRwTHWNuOWu9tGPIGza97PHXFqGW3t377XMsVOdn5IXN/RdtW8y1MqKxWdUHG61Agp OTg3w3hBpZ9uQC2+NZ9h4RDiE4VevF4HmB6gQjazvaE1ERkPalANWrKW7IjswdfBKaG0 C0LA== X-Gm-Message-State: AKwxytfn07aIKoBxLTxxY/EFdJS0NXZFv95XRl2KQhy41Safvqx4XUCg N71IJXWkltXRfxy78o8oH+UllfRuqw4= X-Received: by 10.28.229.194 with SMTP id c185mr12926554wmh.142.1516112627219; Tue, 16 Jan 2018 06:23:47 -0800 (PST) Received: from e108454-lin.cambridge.arm.com ([2001:41d0:1:6c23::1]) by smtp.gmail.com with ESMTPSA id m201sm1686886wma.13.2018.01.16.06.23.45 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 16 Jan 2018 06:23:45 -0800 (PST) From: Julien Grall To: xen-devel@lists.xen.org Date: Tue, 16 Jan 2018 14:23:37 +0000 Message-Id: <20180116142337.24942-6-julien.grall@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180116142337.24942-1-julien.grall@linaro.org> References: <20180116142337.24942-1-julien.grall@linaro.org> Cc: sstabellini@kernel.org, Julien Grall , andre.przywara@linaro.org Subject: [Xen-devel] [PATCH 5/5] xen/arm64: Implement branch predictor hardening for affected Cortex-A CPUs X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" Cortex-A57, A72, A73 and A75 are susceptible to branch predictor aliasing and can theoritically be attacked by malicious code. This patch implements a PSCI-based mitigation for these CPUs when available. The call into firmware will invalidate the branch predictor state, preventing any malicious entries from affection other victim contexts. Ported from Linux git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git branch kpti. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon This is part of XSA-254. Signed-off-by: Julien Grall --- xen/arch/arm/arm64/bpi.S | 25 ++++++++++++++++++++++++ xen/arch/arm/cpuerrata.c | 49 ++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 74 insertions(+) diff --git a/xen/arch/arm/arm64/bpi.S b/xen/arch/arm/arm64/bpi.S index 6cc2f17529..4b7f1dc21f 100644 --- a/xen/arch/arm/arm64/bpi.S +++ b/xen/arch/arm/arm64/bpi.S @@ -56,6 +56,31 @@ ENTRY(__bp_harden_hyp_vecs_start) .endr ENTRY(__bp_harden_hyp_vecs_end) +ENTRY(__psci_hyp_bp_inval_start) + sub sp, sp, #(8 * 18) + stp x16, x17, [sp, #(16 * 0)] + stp x14, x15, [sp, #(16 * 1)] + stp x12, x13, [sp, #(16 * 2)] + stp x10, x11, [sp, #(16 * 3)] + stp x8, x9, [sp, #(16 * 4)] + stp x6, x7, [sp, #(16 * 5)] + stp x4, x5, [sp, #(16 * 6)] + stp x2, x3, [sp, #(16 * 7)] + stp x0, x1, [sp, #(16 * 8)] + mov x0, #0x84000000 + smc #0 + ldp x16, x17, [sp, #(16 * 0)] + ldp x14, x15, [sp, #(16 * 1)] + ldp x12, x13, [sp, #(16 * 2)] + ldp x10, x11, [sp, #(16 * 3)] + ldp x8, x9, [sp, #(16 * 4)] + ldp x6, x7, [sp, #(16 * 5)] + ldp x4, x5, [sp, #(16 * 6)] + ldp x2, x3, [sp, #(16 * 7)] + ldp x0, x1, [sp, #(16 * 8)] + add sp, sp, #(8 * 18) +ENTRY(__psci_hyp_bp_inval_end) + /* * Local variables: * mode: ASM diff --git a/xen/arch/arm/cpuerrata.c b/xen/arch/arm/cpuerrata.c index 76d98e771d..f1ea7f3c5b 100644 --- a/xen/arch/arm/cpuerrata.c +++ b/xen/arch/arm/cpuerrata.c @@ -4,8 +4,10 @@ #include #include #include +#include #include #include +#include /* Override macros from asm/page.h to make them work with mfn_t */ #undef virt_to_mfn @@ -141,6 +143,31 @@ install_bp_hardening_vec(const struct arm_cpu_capabilities *entry, return ret; } +extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; + +static int enable_psci_bp_hardening(void *data) +{ + bool ret = true; + static bool warned = false; + + /* + * The mitigation is using PSCI version function to invalidate the + * branch predictor. This function is only available with PSCI 0.2 + * and later. + */ + if ( psci_ver >= PSCI_VERSION(0, 2) ) + ret = install_bp_hardening_vec(data, __psci_hyp_bp_inval_start, + __psci_hyp_bp_inval_end); + else if ( !warned ) + { + ASSERT(system_state < SYS_STATE_active); + warning_add("PSCI 0.2 or later is required for the branch predictor hardening.\n"); + warned = true; + } + + return !ret; +} + #endif /* CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR */ #define MIDR_RANGE(model, min, max) \ @@ -205,6 +232,28 @@ static const struct arm_cpu_capabilities arm_errata[] = { (1 << MIDR_VARIANT_SHIFT) | 2), }, #endif +#ifdef CONFIG_ARM64_HARDEN_BRANCH_PREDICTOR + { + .capability = ARM_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A73), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A75), + .enable = enable_psci_bp_hardening, + }, +#endif {}, };