From patchwork Mon Feb 26 08:19:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129558 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3355510lja; Mon, 26 Feb 2018 00:21:29 -0800 (PST) X-Google-Smtp-Source: AH8x224vh2WRxjvSkdDab0enxS4uQj8sX64jUWKbU9hMaqLZ4UBIjxs2Ujh2Ctc5TmrjsjKwFUpq X-Received: by 2002:a17:902:ab89:: with SMTP id f9-v6mr9667225plr.369.1519633289039; Mon, 26 Feb 2018 00:21:29 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633289; cv=none; d=google.com; s=arc-20160816; b=N1PT4332nQwaOosslj7pJovcl8Nls4zD0Wh1icgNz/RAwnz0tF/o5s4WmDtKahylfK MDIxCYawOgvJ2S6prPniOzJCKQk6IsfiCAjjnIs2DLvme1LBKFq4JJuZ7FMIcsYab6Pl 8UUrxl/BLA+GbB4KjIq0nD1zVyv/LsveZXt6M/wsKnAc5uMNO/BpK+1Efp1scyUFuraS hIWceynm8tEYJZbSQu9FtHU+GCU0tp3nXA8sH7iNidtue+aIBwad4+3vQIEIFJSr65Ar 99AGgBOv3Rug3knMRDwiwPSR43sRhxylHbCO/EOq4UCjtsL2ReyJUKtqr34NHUhedI2n djLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=S7mDzt0hDdeMXb+Y/dfIW9xk1gio9KtJLmPeqeLlojg=; b=xWnXkSMOHNUihiZSICosJtFZCULh0OtMPz7e6g/QLnqlwFu4YsbjHCbtVmNRqbpCZN 92G4t6FtsNGCR2NQfQ44L0/etseeEUmirsTMUY+0WUTv5mK32NsdYFnDkLQEkOODyt+r by5SjjxIKB+S0efqKTj3FpD2G81RbfzHB/2pBnDqintbMmBIizfNC5uoJYKBVH4gMJxz djYhrDF9jxRvVzuHNHJT7NQqr2u1pYfEvTt9Z+oPerMKLXjhYXniJ4QFoHIZpSJGh04d Yx+3U1+Qp3u2V9UqyiGelUQN99exeYYEeePF4ETcGVR6+djUvPq3PQ431Mu8UqY2qNC0 Ep9g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GfQzyVoC; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j12si5254567pgt.173.2018.02.26.00.21.28; Mon, 26 Feb 2018 00:21:29 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GfQzyVoC; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752209AbeBZIV0 (ORCPT + 28 others); Mon, 26 Feb 2018 03:21:26 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:42661 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751745AbeBZIVV (ORCPT ); Mon, 26 Feb 2018 03:21:21 -0500 Received: by mail-pl0-f67.google.com with SMTP id 31so8853616ple.9 for ; Mon, 26 Feb 2018 00:21:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=S7mDzt0hDdeMXb+Y/dfIW9xk1gio9KtJLmPeqeLlojg=; b=GfQzyVoCstUCriYp03bfIQ1fXM9GSLUPEcJkVcD4xI2yWixaTmK4psqO8k1UFYS6aK XVLXBzUCfvpNv3QYwojgW93iyrHPOhJba6AGE/5ImeKI0S3MlCNW94nY50p1ih6iAaZr eCh9zwMjCiM45BFcxjTqaHsQerXHOimRH8GCQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=S7mDzt0hDdeMXb+Y/dfIW9xk1gio9KtJLmPeqeLlojg=; b=XBF5YThdXOaqjg9ZsHiy5/1Gd8zyF8CYs5s7HcIzmO4qXC+ftK/AoAhYi21ST39GmW 4lvOAj0ImvqcQRPNpcPpTP4ROlMgGnv2CX1/afxiAxZzhuSBQFVFukATvucLLE07TMxg 0cIgS1dbCS0ZDm/mcGVntt3B65yM8488g4wmUqehhDk/TS6cjOq2sfgFTHHqX2jRFV8o u2zUKxrBgjDZMSnQnTBzOOLXMYug+hO/1Tu49adbR2JqjUlOvDpYCadKKJyRNBJ0k5OB wd6b2mevxB4tJADbb76P75OnwL0S3eiGTGvhc8+QX9/urxSLsjbhQciiq0lNDcqGiU67 +fQw== X-Gm-Message-State: APf1xPBTJhm9AbywnixC1h2V+6OzGtfdJAdEAEyt/OCVqlrqlqNC5Euj tDpLidTFFGr3Y1ouHb3xobqvBw== X-Received: by 2002:a17:902:183:: with SMTP id b3-v6mr9789750plb.80.1519633281279; Mon, 26 Feb 2018 00:21:21 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:20 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 02/52] arm64: alternatives: apply boot time fixups via the linear mapping Date: Mon, 26 Feb 2018 16:19:36 +0800 Message-Id: <1519633227-29832-3-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Ard Biesheuvel commit 5ea5306c323 upstream. One important rule of thumb when desiging a secure software system is that memory should never be writable and executable at the same time. We mostly adhere to this rule in the kernel, except at boot time, when regions may be mapped RWX until after we are done applying alternatives or making other one-off changes. For the alternative patching, we can improve the situation by applying the fixups via the linear mapping, which is never mapped with executable permissions. So map the linear alias of .text with RW- permissions initially, and remove the write permissions as soon as alternative patching has completed. Reviewed-by: Laura Abbott Reviewed-by: Mark Rutland Tested-by: Mark Rutland Signed-off-by: Ard Biesheuvel Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: replace update_mapping_prot with old create_mapping_late arch/arm64/mm/mmu.c --- arch/arm64/include/asm/mmu.h | 1 + arch/arm64/kernel/alternative.c | 11 ++++++----- arch/arm64/kernel/smp.c | 1 + arch/arm64/mm/mmu.c | 22 +++++++++++++++++----- 4 files changed, 25 insertions(+), 10 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index 8d9fce0..b075140 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -36,5 +36,6 @@ extern void create_pgd_mapping(struct mm_struct *mm, phys_addr_t phys, unsigned long virt, phys_addr_t size, pgprot_t prot, bool allow_block_mappings); extern void *fixmap_remap_fdt(phys_addr_t dt_phys); +extern void mark_linear_text_alias_ro(void); #endif diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index 06d650f..8840c10 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -105,11 +105,11 @@ static u32 get_alt_insn(struct alt_instr *alt, u32 *insnptr, u32 *altinsnptr) return insn; } -static void __apply_alternatives(void *alt_region) +static void __apply_alternatives(void *alt_region, bool use_linear_alias) { struct alt_instr *alt; struct alt_region *region = alt_region; - u32 *origptr, *replptr; + u32 *origptr, *replptr, *updptr; for (alt = region->begin; alt < region->end; alt++) { u32 insn; @@ -124,11 +124,12 @@ static void __apply_alternatives(void *alt_region) origptr = ALT_ORIG_PTR(alt); replptr = ALT_REPL_PTR(alt); + updptr = use_linear_alias ? (u32 *)lm_alias(origptr) : origptr; nr_inst = alt->alt_len / sizeof(insn); for (i = 0; i < nr_inst; i++) { insn = get_alt_insn(alt, origptr + i, replptr + i); - *(origptr + i) = cpu_to_le32(insn); + updptr[i] = cpu_to_le32(insn); } flush_icache_range((uintptr_t)origptr, @@ -155,7 +156,7 @@ static int __apply_alternatives_multi_stop(void *unused) isb(); } else { BUG_ON(patched); - __apply_alternatives(®ion); + __apply_alternatives(®ion, true); /* Barriers provided by the cache flushing */ WRITE_ONCE(patched, 1); } @@ -176,5 +177,5 @@ void apply_alternatives(void *start, size_t length) .end = start + length, }; - __apply_alternatives(®ion); + __apply_alternatives(®ion, false); } diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c index a70f7d3..66db515 100644 --- a/arch/arm64/kernel/smp.c +++ b/arch/arm64/kernel/smp.c @@ -426,6 +426,7 @@ void __init smp_cpus_done(unsigned int max_cpus) setup_cpu_features(); hyp_mode_check(); apply_alternatives_all(); + mark_linear_text_alias_ro(); } void __init smp_prepare_boot_cpu(void) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index d5cc6d7..5dc72c0 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -354,16 +354,28 @@ static void __init __map_memblock(pgd_t *pgd, phys_addr_t start, phys_addr_t end !debug_pagealloc_enabled()); /* - * Map the linear alias of the [_text, __init_begin) interval as - * read-only/non-executable. This makes the contents of the - * region accessible to subsystems such as hibernate, but - * protects it from inadvertent modification or execution. + * Map the linear alias of the [_text, __init_begin) interval + * as non-executable now, and remove the write permission in + * mark_linear_text_alias_ro() below (which will be called after + * alternative patching has completed). This makes the contents + * of the region accessible to subsystems such as hibernate, + * but protects it from inadvertent modification or execution. */ __create_pgd_mapping(pgd, kernel_start, __phys_to_virt(kernel_start), - kernel_end - kernel_start, PAGE_KERNEL_RO, + kernel_end - kernel_start, PAGE_KERNEL, early_pgtable_alloc, !debug_pagealloc_enabled()); } +void __init mark_linear_text_alias_ro(void) +{ + /* + * Remove the write permissions from the linear alias of .text/.rodata + */ + create_mapping_late(__pa_symbol(_text), (unsigned long)lm_alias(_text), + (unsigned long)__init_begin - (unsigned long)_text, + PAGE_KERNEL_RO); +} + static void __init map_mem(pgd_t *pgd) { struct memblock_region *reg; From patchwork Mon Feb 26 08:19:37 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129559 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3355584lja; Mon, 26 Feb 2018 00:21:36 -0800 (PST) X-Google-Smtp-Source: AH8x2277yj/4RDhanBZ+LyHtB43LK6OqQwb+4tGED4aula1vZjcJkhwu2ltxITIZ1xHM6oZ92kVe X-Received: by 2002:a17:902:a985:: with SMTP id bh5-v6mr9735264plb.230.1519633296105; Mon, 26 Feb 2018 00:21:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633296; cv=none; d=google.com; s=arc-20160816; b=Shk8EUySImMrnZOtzTTmFO1y32BzBfUHTP829ypzWW2FUTGaKGKoafr9/y5JU2JA+K lWRDpSIFxcFG/gs50j8V+fYgTohrOHlYtLnE40KWUYMQUxatR1DeUsho5Iz7Nvn0rkRp v+Io58nXmNmKVEeGRYGgjHhNxjmQ+lI3gHizVMWliwX87ppxBSxRpBN5vZ16BegIgemG hbv8tBgJPe2cfB/gQu/7cXZBrmq3VIJDcOxT1A01lywuMB/j0D5s4D7cd+8ZaCUVtr3O kERM/fjuJUtG7MjoRYrjPVGZiSouPrx6apx2vS16jb3ZyjZNZB5aY9WeV2dT5XXF3fPI 2jKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=5jc9doNqehmWExc3XoeNI1qQ6N7FZfbJliUkP3bf70M=; b=rky8RP6iZf9UlOmt6IEbB4fCDBI9XpoHODCzA0+Y8Np//G9W4j77gpjYCDMvODpDSr ANCAI0kIICRQ8R/4w0hJE3R9bpn9vl9Vzm6wcsn06x6TkF2cS3iqRaZhihDLyhrxKxKA wpa5mS2quak+eKXIWZEeqExMMi16k8J/zQE3s8Zqz33d2ahmFmn5NrrV+9DDXv19PDO/ G2+zdx0TjuhIqOD6en6UhqXMdN4QosSo21HeEjCRTGOZ4IsH0GdN0PPfnQ6oUevwgpSf bwUtblcAzEgjaUi1+hUQ16kCZDS+1gy1jecEJ6mIIo6+VNZVzGxF5mxgFeNi6v+q4tYq aA4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KoMrLdBO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j12si5254567pgt.173.2018.02.26.00.21.35; Mon, 26 Feb 2018 00:21:36 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=KoMrLdBO; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751735AbeBZIVc (ORCPT + 28 others); Mon, 26 Feb 2018 03:21:32 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:38464 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752221AbeBZIV1 (ORCPT ); Mon, 26 Feb 2018 03:21:27 -0500 Received: by mail-pg0-f68.google.com with SMTP id l24so5913590pgc.5 for ; Mon, 26 Feb 2018 00:21:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=5jc9doNqehmWExc3XoeNI1qQ6N7FZfbJliUkP3bf70M=; b=KoMrLdBO5Qvlzer3bGL6wb3OjGdlnUFwBVM2ez8j7L/6ESku7wqpYdcGQuqw9We4Ce ez9+4SLG3bOA+nwgwLrHIgxYaNuGW8orqigbhAuVJ5LeRHV63c4YF5AxJIGlakLPSfwo MzPzkBM5mi/ffzU+X0863/50ZPYaCiNJks25E= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=5jc9doNqehmWExc3XoeNI1qQ6N7FZfbJliUkP3bf70M=; b=MgHnGgbH0iIkHPS/kDH0a82NwVepUg1oEJolIn8F6YH0+GMcbLyHW33AdQlEediPGr 2DZfEv2W48BAWfjiAVsMUZ62VzpDMH5dA21KH7gBsmhRKZ+1+CVu23Y4+WGetcyTrK39 8iU4Fe8Wi+65JjWd+BBtoS3Ovlim02WmSjxP7msKsvWKEyQhP+ii/a+q2VNyhjzkgT8K /V4zkrO9w+jh6IcLJRpqASs0y9XsLBcx7D58cz6pxw1pe5VsYMYtckR6bJGB5yTCos/T WLZPtygtYp/topY1TQbB5fEC2/+HCWuBPv24QMLmV9EVcrtqsR21jCQAMWT8VkI1KPAN gJNQ== X-Gm-Message-State: APf1xPBdpKe60tRQgl8YtU5AnnTisEjPnHNBUNsB8Hn8+RomaOY9A6gM aP+lP541XGsSII9UjrZdtzk6fA== X-Received: by 10.98.226.16 with SMTP id a16mr9691686pfi.157.1519633286699; Mon, 26 Feb 2018 00:21:26 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:26 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 03/52] arm64: barrier: Add CSDB macros to control data-value prediction Date: Mon, 26 Feb 2018 16:19:37 +0800 Message-Id: <1519633227-29832-4-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 669474e772b9 upstream. For CPUs capable of data value prediction, CSDB waits for any outstanding predictions to architecturally resolve before allowing speculative execution to continue. Provide macros to expose it to the arch code. Reviewed-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: arch/arm64/include/asm/assembler.h no psb_csync in arch/arm64/include/asm/barrier.h --- arch/arm64/include/asm/assembler.h | 7 +++++++ arch/arm64/include/asm/barrier.h | 2 ++ 2 files changed, 9 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 851290d..8760300 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -87,6 +87,13 @@ .endm /* + * Value prediction barrier + */ + .macro csdb + hint #20 + .endm + +/* * NOP sequence */ .macro nops, num diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index 0fe7e43..c68fdc5 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -31,6 +31,8 @@ #define dmb(opt) asm volatile("dmb " #opt : : : "memory") #define dsb(opt) asm volatile("dsb " #opt : : : "memory") +#define csdb() asm volatile("hint #20" : : : "memory") + #define mb() dsb(sy) #define rmb() dsb(ld) #define wmb() dsb(st) From patchwork Mon Feb 26 08:19:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129560 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3355716lja; Mon, 26 Feb 2018 00:21:46 -0800 (PST) X-Google-Smtp-Source: AH8x225+3gHoMt1kWsDbszS02Pr3VbP8HAXH6BQ0AZlVRwIEVLojZ7/iY1mPAtPWbEj0NtGS7rT1 X-Received: by 10.99.170.73 with SMTP id x9mr7809734pgo.393.1519633306831; Mon, 26 Feb 2018 00:21:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633306; cv=none; d=google.com; s=arc-20160816; b=rVXx2x3fRKkZrrWUwgXHm5tY8mIcDNt0x+Yhmm+yd7n0L7LHsqPC7xG5Uwpy6DyWWg L5ym7tQGH1bhzwUC64+QpiITK2WV3c+u16lMIu8IAWtpTCAiuQQ91lZhofeiI2A46O3j Onvz2T+bJYl7T25888bjtAc5PxFzChZouNpzooGPnks4jl1R2yaar3UsK+9sX3juDxVL CukMsQaSvzx6ZZfO48U0hHuav1DZ5Vq0hrUT9JUllM7PJ9+SYNxMPAws3JnwNguzJbJq kBZQsc2uVk8VRhGRAT3RBm+5XyJv82uxCh2K/UmpSckdBNwRYe1t4hQLNsEZAZdJvIes fMJQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=1urEszs1X1+fSgF2FtrvQb+AjsP1I5AehCNh9xZoYO0=; b=pc+6jQGS0QV6qCWSaD/OAsavNZMBOpd1R/+UwPL3/9prdkSf1DRdgvJphHtN3Ljj6P yha4KPzSh2rDdsDycSt7VDWOfjSQ87zESY7N101c5nSXaVYeiohe1rRDv61knKDBoccs qATzjWUBF7k97G/XrxKfKUE5TEdDFA4xcRbWIIm6YvBJJFbCPISf1BGI9laPBjQN2aYl gTdo3MSt5gQ1+SKdhAIQzON0h8WE0mf2vkiRFIx2Lo6BPQe3lhBdhGyAAXPt8yqNZ6ld NfpvJv8i2bVwPqFcEZ7Pc33U9FIFu/hPGkXByYGcVp7PmtU6pl+2G9Epojrg3oQpyaC8 8Nbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FZn/iMPF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j12si5255036pgt.173.2018.02.26.00.21.46; Mon, 26 Feb 2018 00:21:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FZn/iMPF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752301AbeBZIVl (ORCPT + 28 others); Mon, 26 Feb 2018 03:21:41 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:42676 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752045AbeBZIVh (ORCPT ); Mon, 26 Feb 2018 03:21:37 -0500 Received: by mail-pl0-f68.google.com with SMTP id 31so8853864ple.9 for ; Mon, 26 Feb 2018 00:21:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1urEszs1X1+fSgF2FtrvQb+AjsP1I5AehCNh9xZoYO0=; b=FZn/iMPFSmQszE4OjKuPqTP8zGFT2OUpn7fMNK+4S7GRPdKcw8xXr9WK6FiV5pRIB7 a9hBBGLrPO2hIANqgHwoKfMhjF1lNDrEA3IdypstLIQtmfnvJTXXKZ37toGohZzN2SOn Ghrm+0JpzirQAWvLfED6dblRBvI11tfIoP2es= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1urEszs1X1+fSgF2FtrvQb+AjsP1I5AehCNh9xZoYO0=; b=qWk/1U9S7TnRmPi8puD4MKEtlV5MlVroq0/yS5WgFC44gUTgxeF9vvAMfnL6w0HZmm QYErPoICONvpCaDNRBbHg2e9V3LvqQ7SN1HiPg/dTXdhEcYuraUg7k3Kb60wTOBpywV2 Z3miid3Bk/RnCVFAcFvcnUFVt0zOENpn3YnYstyBqqUUyOklk8cC98HQr97bWno57+hK 1YcoW9hbvUJ3Ii9uUwMre/ULlG95pp9ny3P3OXsBW1HfMG42K+c74NE7rMAHbsLDphr3 yag8npqqVdUu5yG9i1EtmbRXlFnHIj3mrViJaoKk5H19UJgu2eM9lVO+Qj02kvHG0KWE l93A== X-Gm-Message-State: APf1xPBkpgBa0yKkWtxcoiO5GAtrfUXpBTscv6U2t55DYwqQeKrNKWVz 81BqYKvGULAchU3sZE06ws2zTQ== X-Received: by 2002:a17:902:5e3:: with SMTP id f90-v6mr9940812plf.413.1519633296241; Mon, 26 Feb 2018 00:21:36 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.27 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:35 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Robin Murphy Subject: [PATCH 04/52] arm64: Implement array_index_mask_nospec() Date: Mon, 26 Feb 2018 16:19:38 +0800 Message-Id: <1519633227-29832-5-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 022620eed3d0 upstream. Provide an optimised, assembly implementation of array_index_mask_nospec() for arm64 so that the compiler is not in a position to transform the code in ways which affect its ability to inhibit speculation (e.g. by introducing conditional branches). This is similar to the sequence used by x86, modulo architectural differences in the carry/borrow flags. Reviewed-by: Mark Rutland Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/barrier.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/barrier.h b/arch/arm64/include/asm/barrier.h index c68fdc5..0b0755c 100644 --- a/arch/arm64/include/asm/barrier.h +++ b/arch/arm64/include/asm/barrier.h @@ -40,6 +40,27 @@ #define dma_rmb() dmb(oshld) #define dma_wmb() dmb(oshst) +/* + * Generate a mask for array_index__nospec() that is ~0UL when 0 <= idx < sz + * and 0 otherwise. + */ +#define array_index_mask_nospec array_index_mask_nospec +static inline unsigned long array_index_mask_nospec(unsigned long idx, + unsigned long sz) +{ + unsigned long mask; + + asm volatile( + " cmp %1, %2\n" + " sbc %0, xzr, xzr\n" + : "=r" (mask) + : "r" (idx), "Ir" (sz) + : "cc"); + + csdb(); + return mask; +} + #define __smp_mb() dmb(ish) #define __smp_rmb() dmb(ishld) #define __smp_wmb() dmb(ishst) From patchwork Mon Feb 26 08:19:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129561 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3355805lja; Mon, 26 Feb 2018 00:21:53 -0800 (PST) X-Google-Smtp-Source: AH8x226MaoaKmHZRKJY3nvhpIgNdHT6SSkx7pzPOrDGOn5vxiDY/STgTzmgNqMtHVqH98Ail3zLh X-Received: by 10.101.81.132 with SMTP id h4mr7902607pgq.332.1519633313208; Mon, 26 Feb 2018 00:21:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633313; cv=none; d=google.com; s=arc-20160816; b=xxevric9ok0indKYUCm32mO2o3habHx+Ga/o1Y5KqhwnVBXs3xksmUKtv79Jei2n5T 0BVTQq52JogSD5kax7LxIKdfrlojrpzDS41niPoEAs6KTi6OoWJfN4iOl/FtE42MIppi mlIcFz2JRK1Ua7PUPGOzuX0ksThomT/GMJPK6YJQksiNhw6ZoKYRq+JPXt2qWkLaGSn8 SypA4VpjPan4nC49nugSfONDu5LUvOWRTFBPh7MNh+cOh4bFjfZkmmP0sNMyMBlVwwRm ouO5U7fYIqcBtqR3bzvKQ/tQLO4o9au9nEuNmU8DEPFSO0qNFl7gx4G+C/hltOc05yHr eV9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=5lK+ndVcYpPWXEM1yPl0Iv5G8Ce26XZw/h8DlWl28GE=; b=0EJcydULThujJ+r/x++eyngeEYGQ5vPbx2l9Y0r6wnt2WA97W/z40J065rTMwOhFtG RWhaEuX+To3sLZcq0LuS79OCxBvqmYZiyWSQ3wdwt8BySewV6Raa3yNHYhxfjs72Lll8 C/GuN4bimedIRKJ6++WQTRjdRP2A9aVmHXUz1SSPsnYgghz5McvuVb6XmYNvMlzzmhgM mehIBDvTmZlTnMjoceqlYfenSSnUz8SlutGT3dqT3r2M8VnuE6+XGiXkS9NfHR1J8ONo mK0h0NJLJiIWeT0ctyrA+Q+1YIH21hfQYEXmovodsigGig9byie0h+kgBMjtV91NywF4 gjiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QGcL7VmY; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y8si5246129pgp.602.2018.02.26.00.21.52; Mon, 26 Feb 2018 00:21:53 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QGcL7VmY; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752330AbeBZIVt (ORCPT + 28 others); Mon, 26 Feb 2018 03:21:49 -0500 Received: from mail-pl0-f67.google.com ([209.85.160.67]:45598 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752310AbeBZIVo (ORCPT ); Mon, 26 Feb 2018 03:21:44 -0500 Received: by mail-pl0-f67.google.com with SMTP id v9-v6so6856828plp.12 for ; Mon, 26 Feb 2018 00:21:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5lK+ndVcYpPWXEM1yPl0Iv5G8Ce26XZw/h8DlWl28GE=; b=QGcL7VmYMOjABfHXklK5mYK9Dx7YInSEoSATxs//pCEz7WUGTamW7UV+x66x9HSRMw iauYrcy/MXQzEOYauWTMvPW9VusudthSGjfMtu74XEqMDdgR9OF6Tsy7dDy69BTQNYhp 1xp/MLFRWjS8cdCmBjPFO4PWd/ONRBVPz2e0I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5lK+ndVcYpPWXEM1yPl0Iv5G8Ce26XZw/h8DlWl28GE=; b=caqr8pft2B6tGSN7hh68e+c/twsTJtNBnkpxhvJOSrYOdacXHz4QwhwRfmxm+Iphvc PjDJM0csnou0CuOoD34/jM9ftHsFhLn1cx1iFZQAlfXkyhyO2Rh7+soIQbPPiIue8M7e KbD+dxyNFaY1YA6DrBcqv3luIZImxHDEKu+saVeHpv/rVa7gYgaPWNUKLT5AVkvse25D I0ZHFBVnmhe3IPR+o+rsjX4EhlDxO1a5FXvo1SgxvEqGETUIrbHJt3Y7RPNo/CcPUx3e ib6mdxT2gMiSMYs4Ghg58YaKWyKApMgx6XBsnNbqXFb5ZKINWpCrKlwxKD7CU8XXhAl7 f1bA== X-Gm-Message-State: APf1xPDpB9t3f2InltLUG0VS27Ykcu8SbmvqmSa4LqHcjNSGBfsyT5Oo nGPyOENzTnKPRkUUNH9xffcUOg== X-Received: by 2002:a17:902:7796:: with SMTP id o22-v6mr10074745pll.215.1519633303792; Mon, 26 Feb 2018 00:21:43 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:21:43 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Yury Norov , Laura Abbott , James Morse Subject: [PATCH 05/52] arm64: move TASK_* definitions to Date: Mon, 26 Feb 2018 16:19:39 +0800 Message-Id: <1519633227-29832-6-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Yury Norov commit eef94a3d09aab upstream. ILP32 series [1] introduces the dependency on for TASK_SIZE macro. Which in turn requires , and include , giving a circular dependency, because TASK_SIZE is currently located in . In other architectures, TASK_SIZE is defined in , and moving TASK_SIZE there fixes the problem. Discussion: https://patchwork.kernel.org/patch/9929107/ [1] https://github.com/norov/linux/tree/ilp32-next CC: Will Deacon CC: Laura Abbott Cc: Ard Biesheuvel Cc: Catalin Marinas Cc: James Morse Suggested-by: Mark Rutland Signed-off-by: Yury Norov Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no ptrace.h in arch/arm64/kernel/entry.S --- arch/arm64/include/asm/memory.h | 15 --------------- arch/arm64/include/asm/processor.h | 21 +++++++++++++++++++++ arch/arm64/kernel/entry.S | 2 +- 3 files changed, 22 insertions(+), 16 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 53211a0..269b979 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -60,8 +60,6 @@ * KIMAGE_VADDR - the virtual address of the start of the kernel image * VA_BITS - the maximum number of bits for virtual addresses. * VA_START - the first kernel virtual address. - * TASK_SIZE - the maximum size of a user space task. - * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area. */ #define VA_BITS (CONFIG_ARM64_VA_BITS) #define VA_START (UL(0xffffffffffffffff) << VA_BITS) @@ -74,19 +72,6 @@ #define PCI_IO_END (VMEMMAP_START - SZ_2M) #define PCI_IO_START (PCI_IO_END - PCI_IO_SIZE) #define FIXADDR_TOP (PCI_IO_START - SZ_2M) -#define TASK_SIZE_64 (UL(1) << VA_BITS) - -#ifdef CONFIG_COMPAT -#define TASK_SIZE_32 UL(0x100000000) -#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \ - TASK_SIZE_32 : TASK_SIZE_64) -#define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \ - TASK_SIZE_32 : TASK_SIZE_64) -#else -#define TASK_SIZE TASK_SIZE_64 -#endif /* CONFIG_COMPAT */ - -#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4)) #define KERNEL_START _text #define KERNEL_END _end diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 60e3482..4258f4d 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -19,6 +19,10 @@ #ifndef __ASM_PROCESSOR_H #define __ASM_PROCESSOR_H +#define TASK_SIZE_64 (UL(1) << VA_BITS) + +#ifndef __ASSEMBLY__ + /* * Default implementation of macro that returns current * instruction pointer ("program counter"). @@ -37,6 +41,22 @@ #include #include +/* + * TASK_SIZE - the maximum size of a user space task. + * TASK_UNMAPPED_BASE - the lower boundary of the mmap VM area. + */ +#ifdef CONFIG_COMPAT +#define TASK_SIZE_32 UL(0x100000000) +#define TASK_SIZE (test_thread_flag(TIF_32BIT) ? \ + TASK_SIZE_32 : TASK_SIZE_64) +#define TASK_SIZE_OF(tsk) (test_tsk_thread_flag(tsk, TIF_32BIT) ? \ + TASK_SIZE_32 : TASK_SIZE_64) +#else +#define TASK_SIZE TASK_SIZE_64 +#endif /* CONFIG_COMPAT */ + +#define TASK_UNMAPPED_BASE (PAGE_ALIGN(TASK_SIZE / 4)) + #define STACK_TOP_MAX TASK_SIZE_64 #ifdef CONFIG_COMPAT #define AARCH32_VECTORS_BASE 0xffff0000 @@ -192,4 +212,5 @@ int cpu_enable_pan(void *__unused); int cpu_enable_uao(void *__unused); int cpu_enable_cache_maint_trap(void *__unused); +#endif /* __ASSEMBLY__ */ #endif /* __ASM_PROCESSOR_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index b4c7db4..478f0fe 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -28,7 +28,7 @@ #include #include #include -#include +#include #include #include #include From patchwork Mon Feb 26 08:19:42 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129564 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356018lja; Mon, 26 Feb 2018 00:22:13 -0800 (PST) X-Google-Smtp-Source: AH8x226b2SWUkpw77Kx6d9f5Hgnxgktjq5v7hcWSpA9Y+6q5lDV9b8ZDMmhroR/3TpeBK55IWMTT X-Received: by 10.99.117.26 with SMTP id q26mr8100339pgc.46.1519633333464; Mon, 26 Feb 2018 00:22:13 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633333; cv=none; d=google.com; s=arc-20160816; b=E+no2AcXDt8v96EQyZ4Y45fXoUzdep2PIgIxAzI48qZyqh00MDRkx2B09PA7nRlIMV ykzZr7YoNCLrLIglgdXMB/GFHUqHRdPMTm9Tm7QDNtaCI7eNx9G0uBwkPe73BKENQIZ5 +ocjFnIacYDLR8IAFVT0T4zJiwWjHHXLThzZEPr0Ju1HXCG3f46k7jQDsbxChQeELhSU vtPGEWiByxp5uHLuNvXqnQtw3hcJfz8egCasDr9DjpRLFsHOaEKjPpLSCW8DugTYQ6Su u4WawwkOWqVGzO4Iw4PO+uGvBOjxbxt5F22ivmXG5N1lGROTdHFGHieYqwh9MFyilDS5 kb9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=faQj8JqC4/KwGNZMuLXKQ1k5PdvnOQTh5XG4tCkdUZY=; b=KnH4LJL5dQIVe0AplOqg2lO/5YzJsBk5bgxg5LJVktlSNvc5z4mmHC4EImfhIqienY 3CPnvFkQGzjhHDHT/Nfge9N7f/0dZao0yHzsklHrFV/No3M/pfnqPWts8gHHw2f1xQnr DpKkL81+5VwsMfpR+5NgmJhksvt2LgkdSDgY/+DI+Yac7yVca8WyMYiJc5G519b5CuH8 Sn1nozAy35v2WCR+AdNlj0sNufKq/K/jZcc18qLKqReDEZ0HusHz0vCgWzP2xq/fyn3F y4+D+AIt0hZ6QIcR3eP0XLPbqnrLFfGUrimCDLaZcu1HrJlDCIqFLDFOgwyDmJ9rEXxR YEFg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CnxY6PLp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id o5si1549239pfh.390.2018.02.26.00.22.13; Mon, 26 Feb 2018 00:22:13 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CnxY6PLp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752386AbeBZIWJ (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:09 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:45434 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752331AbeBZIWF (ORCPT ); Mon, 26 Feb 2018 03:22:05 -0500 Received: by mail-pf0-f194.google.com with SMTP id j24so6215839pff.12 for ; Mon, 26 Feb 2018 00:22:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=faQj8JqC4/KwGNZMuLXKQ1k5PdvnOQTh5XG4tCkdUZY=; b=CnxY6PLpkO4tHJqWj2rQ4/NjiX0Nxnxgf9etNtMJ9uCyXyiUYyrXV1usAmq5stT+dA OmpMx/YgVPBIXIGZg7MmF92x2zdDLRN64ErpGZmyl5veaYxaGn/vE80bXQzlzqIVdz3z c+Q+qT/H9t+zRZKd0Vln9N3uqdhAAHsb3nXuE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=faQj8JqC4/KwGNZMuLXKQ1k5PdvnOQTh5XG4tCkdUZY=; b=ZoA/n831DmEIOGKy4+zgJ9UM+qiATThyR+bih97acFMtgS8e9+dFJA0+Cik6h8Bw9C yIVcHcfp+sNu3Tx+kUJo9R65WspvpYgUXDGBKBCO0L7VEjOrJU61Gm0fDplc0RThqtZ6 PKSjzG7tSbCYI0SwQ3sU1kkRex1ug+8aCV6VLpaS5SKBa7nRmHz7mZXUuJ4Ooqig+4Jg Kgq4CEy3KSBN+DWDFUmdfLhO7VFQ1WD8gBp06FN4TNhq5OH2GADsxrG//A6XQa9w0jGu RCGWZGMkMhZ3txrix0Y5gNY9dp+NGeZ5sPwW5mhXAnIQGXDmyUSUGutSD2dy9AsDCBPR z5VA== X-Gm-Message-State: APf1xPBo77zbxi0HxGIw0vVz9SmCZn7EC5RtZ73k92u9EfHvXsUqcqNW jKk/VJzGOZ5mDsgnCTWNrsNnDw== X-Received: by 10.99.4.197 with SMTP id 188mr7595547pge.359.1519633325303; Mon, 26 Feb 2018 00:22:05 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.21.59 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:04 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Mark Rutland Subject: [PATCH 08/52] arm64: uaccess: consistently check object sizes Date: Mon, 26 Feb 2018 16:19:42 +0800 Message-Id: <1519633227-29832-9-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mark Rutland commit 76624175dca upstream. Currently in arm64's copy_{to,from}_user, we only check the source/destination object size if access_ok() tells us the user access is permissible. However, in copy_from_user() we'll subsequently zero any remainder on the destination object. If we failed the access_ok() check, that applies to the whole object size, which we didn't check. To ensure that we catch that case, this patch hoists check_object_size() to the start of copy_from_user(), matching __copy_from_user() and __copy_to_user(). To make all of our uaccess copy primitives consistent, the same is done to copy_to_user(). Cc: Catalin Marinas Acked-by: Kees Cook Signed-off-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/include/asm/uaccess.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 94e1457..09c9b59 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -326,9 +326,9 @@ static inline unsigned long __must_check copy_from_user(void *to, const void __u { unsigned long res = n; kasan_check_write(to, n); + check_object_size(to, n, false); if (access_ok(VERIFY_READ, from, n)) { - check_object_size(to, n, false); res = __arch_copy_from_user(to, from, n); } if (unlikely(res)) @@ -339,9 +339,9 @@ static inline unsigned long __must_check copy_from_user(void *to, const void __u static inline unsigned long __must_check copy_to_user(void __user *to, const void *from, unsigned long n) { kasan_check_read(from, n); + check_object_size(from, n, true); if (access_ok(VERIFY_WRITE, to, n)) { - check_object_size(from, n, true); n = __arch_copy_to_user(to, from, n); } return n; From patchwork Mon Feb 26 08:19:43 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129565 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356100lja; Mon, 26 Feb 2018 00:22:21 -0800 (PST) X-Google-Smtp-Source: AH8x226lDyaqoHCcVPvdGsnJ/UyrxMEoxZtw3Le6U7gH1UsGBEHNjkSxe0USYbroUai+WWvwkd4g X-Received: by 10.99.103.69 with SMTP id b66mr7760843pgc.233.1519633341118; Mon, 26 Feb 2018 00:22:21 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633341; cv=none; d=google.com; s=arc-20160816; b=P5XYF4zbBk32z7LqWvUKytC2vXg+LQgODjjLzlXp0jDn9hTzl9JIcLq4GF/Drvtf2L 7Pv5c3yilFfvMAijJ+Zaz7xp5X30kUhd5PP85BeS+Y05r5En0BsGZLXwBwYzENyxASEU YMfBEGu2+pv5YAA6PtfiBAGuh6AGROIqe+IztB/7dKjxpAgJmwv4tYqdMS4v8CHhgFAd V3X+18WNCqp9iUg5/U4uiXhDqwMe6WrL9dU5cTFfj5LSqr7AF6ulns9FpE906jCaA5y8 tHiIWU2kyEMEAU7O1bbHoEWQ+Hws72CleKXWeet4JJczM75Cf2Iak3GKclQhsnQQOX9R l3EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=lhAelKOq2c/bCw+d6iNI4ebwtqsTn1Sf3YKBl7GlNsA=; b=kLvfmXnWmAcTLHO4SHsgjaJnxGwZiMZ6wnkD36Xy92o9GPt9+CfE818VRSpQXwpaBq 1kWtrGDWak2O7dQCvP9Y58YHgqUo0d0icIzvcG4venuFVN83xM5sYkx7iUen3Jdj7q/O Mxz1qdAn0dEAboWiExFOh9IzyNHSCXp4UD2+O+Vw8nZS2hyLn/8yfnSv7nv+ot+98+2o US6qKtHawI5mcPzSdI4eIW6CasuXtVKkp0PUNF3friDuyKH/s6ikklbBo+T8ViGphLTm Mh5eSkQbRPPxM1vLpvGdaAx7NYqCHGI5bFD2qtTwVM9EJt7OwJR9q6CdKboNOiaIwuij CSNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=khZZHeON; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x13si5260385pgc.75.2018.02.26.00.22.20; Mon, 26 Feb 2018 00:22:21 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=khZZHeON; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752400AbeBZIWR (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:17 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:38381 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751530AbeBZIWN (ORCPT ); Mon, 26 Feb 2018 03:22:13 -0500 Received: by mail-pf0-f193.google.com with SMTP id d26so6219694pfn.5 for ; Mon, 26 Feb 2018 00:22:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=lhAelKOq2c/bCw+d6iNI4ebwtqsTn1Sf3YKBl7GlNsA=; b=khZZHeONaEEupHkBSAr5hTKGNyxnUDb3a2umThtM0LhaYWZOKFVGPinL6RU3KO3BRi 2F0QIoG0h5jOWaIn3WjeHBbgGDrKPbyJm4b1eu2Qf0Fd/+cAcY8Jthvfu8d2JrLaKTpT MJwuW/6sKG15JgkTAGO8lhbbXlI6eB4nMuWdM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=lhAelKOq2c/bCw+d6iNI4ebwtqsTn1Sf3YKBl7GlNsA=; b=AHB/GNf+oLVhT35ZGWxXGHwP1ajqRw2Ptsf9VKEsdB/cNogdYtmwop1yCuLp5JkDmu 8R6+IOH1JRDPt0xdS9D9dAwW6wpZiC2ZRB2PDlmY0q0YcwS1FgixKQrcEKz7BazCINoI e7UBenqKwZ8Soav6G5P9TTkoRXReHRvYq2ZSa9bRgpk9qjerIf/JSLH/AUUQZM7XTp0S gmGpdJReQzVa2CiSspTw8Gxa7B5sXjbvFc5WrMWpk+CEiHNqNjMjtxgQ9xO8SwXDv0vG yRc9GPolqFfChUV7lZNkfDRVhHUhPUw30m/ixe1Eiec1atRij9ygXKNVRUnsUoe8OpZX ekkg== X-Gm-Message-State: APf1xPAusCeiHYmzxu2lq71ZVhHz4tK5+xRPSDZtIUzVlPY0ch/uPDeV ps82wu1Jc91H+CciyuFlw9Cd7Q== X-Received: by 10.98.98.194 with SMTP id w185mr9862265pfb.9.1519633332837; Mon, 26 Feb 2018 00:22:12 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:12 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Robin Murphy Subject: [PATCH 09/52] arm64: Make USER_DS an inclusive limit Date: Mon, 26 Feb 2018 16:19:43 +0800 Message-Id: <1519633227-29832-10-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 51369e398d0d upstream. Currently, USER_DS represents an exclusive limit while KERNEL_DS is inclusive. In order to do some clever trickery for speculation-safe masking, we need them both to behave equivalently - there aren't enough bits to make KERNEL_DS exclusive, so we have precisely one option. This also happens to correct a longstanding false negative for a range ending on the very top byte of kernel memory. Mark Rutland points out that we've actually got the semantics of addresses vs. segments muddled up in most of the places we need to amend, so shuffle the {USER,KERNEL}_DS definitions around such that we can correct those properly instead of just pasting "-1"s everywhere. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas (cherry picked from commit 83b20dff71ea949431cf57c6aebaaf7ebd5c1991) Signed-off-by: Alex Shi Conflicts: force replace __range_ok and add asm/processor.h in arch/arm64/include/asm/uaccess.h using old macro TI_ADDR_LIMIT instead of TSK_TI_ADDR_LIMIT in arch/arm64/kernel/entry.S manual change USER_DS to TASK_SIZE in arch/arm64/mm/fault.c --- arch/arm64/include/asm/processor.h | 3 +++ arch/arm64/include/asm/uaccess.h | 46 ++++++++++++++++++++++---------------- arch/arm64/kernel/entry.S | 2 +- arch/arm64/mm/fault.c | 2 +- 4 files changed, 32 insertions(+), 21 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 4258f4d..5917147 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -21,6 +21,9 @@ #define TASK_SIZE_64 (UL(1) << VA_BITS) +#define KERNEL_DS UL(-1) +#define USER_DS (TASK_SIZE_64 - 1) + #ifndef __ASSEMBLY__ /* diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 09c9b59..7b1eb49 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -36,6 +36,7 @@ #include #include #include +#include #define VERIFY_READ 0 #define VERIFY_WRITE 1 @@ -62,10 +63,7 @@ struct exception_table_entry extern int fixup_exception(struct pt_regs *regs); -#define KERNEL_DS (-1UL) #define get_ds() (KERNEL_DS) - -#define USER_DS TASK_SIZE_64 #define get_fs() (current_thread_info()->addr_limit) static inline void set_fs(mm_segment_t fs) @@ -90,22 +88,32 @@ static inline void set_fs(mm_segment_t fs) * Returns 1 if the range is valid, 0 otherwise. * * This is equivalent to the following test: - * (u65)addr + (u65)size <= current->addr_limit - * - * This needs 65-bit arithmetic. + * (u65)addr + (u65)size <= (u65)current->addr_limit + 1 */ -#define __range_ok(addr, size) \ -({ \ - unsigned long __addr = (unsigned long __force)(addr); \ - unsigned long flag, roksum; \ - __chk_user_ptr(addr); \ - asm("adds %1, %1, %3; ccmp %1, %4, #2, cc; cset %0, ls" \ - : "=&r" (flag), "=&r" (roksum) \ - : "1" (__addr), "Ir" (size), \ - "r" (current_thread_info()->addr_limit) \ - : "cc"); \ - flag; \ -}) +static inline unsigned long __range_ok(unsigned long addr, unsigned long size) +{ + unsigned long limit = current_thread_info()->addr_limit; + + __chk_user_ptr(addr); + asm volatile( + // A + B <= C + 1 for all A,B,C, in four easy steps: + // 1: X = A + B; X' = X % 2^64 + " adds %0, %0, %2\n" + // 2: Set C = 0 if X > 2^64, to guarantee X' > C in step 4 + " csel %1, xzr, %1, hi\n" + // 3: Set X' = ~0 if X >= 2^64. For X == 2^64, this decrements X' + // to compensate for the carry flag being set in step 4. For + // X > 2^64, X' merely has to remain nonzero, which it does. + " csinv %0, %0, xzr, cc\n" + // 4: For X < 2^64, this gives us X' - C - 1 <= 0, where the -1 + // comes from the carry in being clear. Otherwise, we are + // testing X' - C == 0, subject to the previous adjustments. + " sbcs xzr, %0, %1\n" + " cset %0, ls\n" + : "+r" (addr), "+r" (limit) : "Ir" (size) : "cc"); + + return addr; +} /* * When dealing with data aborts, watchpoints, or instruction traps we may end @@ -114,7 +122,7 @@ static inline void set_fs(mm_segment_t fs) */ #define untagged_addr(addr) sign_extend64(addr, 55) -#define access_ok(type, addr, size) __range_ok(addr, size) +#define access_ok(type, addr, size) __range_ok((unsigned long)(addr), size) #define user_addr_max get_fs #define _ASM_EXTABLE(from, to) \ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 478f0fe..6915697 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -103,7 +103,7 @@ /* Save the task's original addr_limit and set USER_DS (TASK_SIZE_64) */ ldr x20, [tsk, #TI_ADDR_LIMIT] str x20, [sp, #S_ORIG_ADDR_LIMIT] - mov x20, #TASK_SIZE_64 + mov x20, #USER_DS str x20, [tsk, #TI_ADDR_LIMIT] /* No need to reset PSTATE.UAO, hardware's already set it to 0 for us */ .endif /* \el == 0 */ diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 403fe9e..4df70c9 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -332,7 +332,7 @@ static int __kprobes do_page_fault(unsigned long addr, unsigned int esr, mm_flags |= FAULT_FLAG_WRITE; } - if (is_permission_fault(esr) && (addr < USER_DS)) { + if (is_permission_fault(esr) && (addr < TASK_SIZE)) { /* regs->orig_addr_limit may be 0 if we entered from EL0 */ if (regs->orig_addr_limit == KERNEL_DS) die("Accessing user space memory with fs=KERNEL_DS", regs, esr); From patchwork Mon Feb 26 08:19:44 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129566 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356190lja; Mon, 26 Feb 2018 00:22:31 -0800 (PST) X-Google-Smtp-Source: AH8x2248pImNHeZr5HhKkJJgfB+zItBfZ2br2xHcnuD9QyJp+yvWB4OaaYpmkdTLXgwvFPAZyoXS X-Received: by 10.99.120.197 with SMTP id t188mr7788786pgc.358.1519633351028; Mon, 26 Feb 2018 00:22:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633351; cv=none; d=google.com; s=arc-20160816; b=GndVuyBDcbWhI5N+PU3UDLTRgeVHgdm6w2G3BKPI6vsBZ+FI6ufVP4crQpdF4GcNeG dmNAPcH6FQ/dA9fRnxdeEehL8Q0QiD1BLb42HyDIRtu8JX1SLzGjzDMlF6V9sufBe7K7 J+FGdeYhbziAUt8Af+/vdNN7YZfsCjge4CnDddRuQ7kYXWgjpQBnqQTSxO0nZxN3A0BP lwSlihzO0PCjA3nTQVYjWjj+4uC3xVbccotKG0M5G79A+gIO9CNY2vosAEdgF/sR67T7 AWMabqgocqKwEjXycx97mS9UqBMlTYdDaeCAqIrfOQFyuDSvjFdG5Sl3dG8NTcDgWZot yh2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=i4j5LLlcJTrNi6a03osuM7FMBFfhyS0MAdksfWz86sU=; b=SgdfdeO1R4gRRAdxmZe8goPlFe/t0y7SeefQ//TQw9CHXHUUeUv5poU1kCZouA2Tgs G0G4s8dTablusZHpdxxATC+LpP2+JPHRH5P9rVy7ahMv5BJnju1ptUbU5BustEtk1ITF qndow0SbnthMWBTrCD65mJCC6VB+3yatC8DMZayk7W+CnY1T/4RoPb/p1V0Ml8o4nKFq R5ynLMc4P+ZqmzD5wuFQmSySR0A177/kztdII6Lvl4YDmFbin5Hu8a/at+1VZbUa6k5K RokPX/3ruGUEWzxe2eLG1NcYgTWyjWv1nwcPqfoO4YG7cEAMyvF7LwqGf7c/9GxMpxuS x8qA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JAXXbMCF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q3si5259730pgp.632.2018.02.26.00.22.30; Mon, 26 Feb 2018 00:22:31 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JAXXbMCF; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752418AbeBZIW1 (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:27 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:34067 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752355AbeBZIWU (ORCPT ); Mon, 26 Feb 2018 03:22:20 -0500 Received: by mail-pg0-f66.google.com with SMTP id m19so5916951pgn.1 for ; Mon, 26 Feb 2018 00:22:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=i4j5LLlcJTrNi6a03osuM7FMBFfhyS0MAdksfWz86sU=; b=JAXXbMCFb1ywC7DpCbKF9Md+/6NZLBeBNU/5p9nQW44JJHgLVsijAuPprSsoWWT8r8 Iez3tDbLbtVRIlMjyyubTxKSFyo1KXxFHWg9bs9MiCS8MP60KANfh02NIvU40WIgXj3u u3o4rkybMjWSWGC0O3qTMC+AurxbDZkQL15Xs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=i4j5LLlcJTrNi6a03osuM7FMBFfhyS0MAdksfWz86sU=; b=EhrfEq6pWzIuiYOs6psakJT2vPhY6FtHP/9gZVCM0hGmD8+0ZfyBgJGEK3Nj5S/TP9 8aRl8k66HS0TBHy2iPgozlMNQjjjT/Nho2MFL4jbQqL0oJI0hwutIt3t4SulsgJ/EmDr P1YMrfiu12f2dlIn0N6VmW2I7LX0PThmrX8JXWSb06G2vBCIimlusaV48ag9upTT9Z0Q hAo9sEyWRGZYlgrXbWbLAeFfcnk01eqPJ4GzrqT/RCi6+UvsiiCtktR+o2jcvPHm1FiT RvVLcl45fnfQEeRYs64k1BbZBlcnI45omrhmSrlkL5qdI1DLd4lNmZR/xMPA/v+R71d8 u1uA== X-Gm-Message-State: APf1xPDsxeFFp+6ap3r7YaAC/vcJeq0u+KlON+ZkUh9FqbNrtgY1RHed I0kL/Y9acl3UbpCLEwIE617gOA== X-Received: by 10.101.99.205 with SMTP id n13mr7925896pgv.345.1519633339675; Mon, 26 Feb 2018 00:22:19 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:19 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Robin Murphy Subject: [PATCH 10/52] arm64: Use pointer masking to limit uaccess speculation Date: Mon, 26 Feb 2018 16:19:44 +0800 Message-Id: <1519633227-29832-11-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Robin Murphy commit 4d8efc2d5ee4 upstream. Similarly to x86, mitigate speculation past an access_ok() check by masking the pointer against the address limit before use. Even if we don't expect speculative writes per se, it is plausible that a CPU may still speculate at least as far as fetching a cache line for writing, hence we also harden put_user() and clear_user() for peace of mind. Signed-off-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas --- arch/arm64/include/asm/uaccess.h | 26 +++++++++++++++++++++++--- 1 file changed, 23 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 7b1eb49..3531fec 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -170,6 +170,26 @@ static inline void uaccess_enable_not_uao(void) } /* + * Sanitise a uaccess pointer such that it becomes NULL if above the + * current addr_limit. + */ +#define uaccess_mask_ptr(ptr) (__typeof__(ptr))__uaccess_mask_ptr(ptr) +static inline void __user *__uaccess_mask_ptr(const void __user *ptr) +{ + void __user *safe_ptr; + + asm volatile( + " bics xzr, %1, %2\n" + " csel %0, %1, xzr, eq\n" + : "=&r" (safe_ptr) + : "r" (ptr), "r" (current_thread_info()->addr_limit) + : "cc"); + + csdb(); + return safe_ptr; +} + +/* * The "__xxx" versions of the user access functions do not verify the address * space - it must have been done previously with a separate "access_ok()" * call. @@ -241,7 +261,7 @@ do { \ __typeof__(*(ptr)) __user *__p = (ptr); \ might_fault(); \ access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \ - __get_user((x), __p) : \ + __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \ ((x) = 0, -EFAULT); \ }) @@ -307,7 +327,7 @@ do { \ __typeof__(*(ptr)) __user *__p = (ptr); \ might_fault(); \ access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \ - __put_user((x), __p) : \ + __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \ -EFAULT; \ }) @@ -368,7 +388,7 @@ static inline unsigned long __must_check copy_in_user(void __user *to, const voi static inline unsigned long __must_check clear_user(void __user *to, unsigned long n) { if (access_ok(VERIFY_WRITE, to, n)) - n = __clear_user(to, n); + n = __clear_user(__uaccess_mask_ptr(to), n); return n; } From patchwork Mon Feb 26 08:19:45 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129567 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356283lja; Mon, 26 Feb 2018 00:22:39 -0800 (PST) X-Google-Smtp-Source: AH8x224GQGdLgmw+oZ2KG0xTxMBfda70HryKmJSrgRKCcOvYghIhrA9PMqppYI87u4cp1JyjPgvW X-Received: by 2002:a17:902:9a04:: with SMTP id v4-v6mr9699181plp.252.1519633359231; Mon, 26 Feb 2018 00:22:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633359; cv=none; d=google.com; s=arc-20160816; b=0c1N3GGrp6vLc1oOUboh4J9G8CQ3pjtYHKpMTrFAh+pEystfsBXxwfNaSwUu4rWcqC WAt9fwv5PZLsIw3M8ofvr7B29s2T09m8azkPv7OJT3np+vxH4JoT7hiTXrwZc8RkhWNS N4JHR6oRmt0Aeai7YXEziE7+oaIwPF0scCwkBfatktmQ3SSmefHrmTVfQJKnWaOnFk0n K7M2n1hcFK9NIbSi0i5FeDfURU27E65o52KuG0wm06gY18iMtM6AsnkjkxHubZU2nufr R/zB6SOz2vxSRjbJ8cz0RH3+U1MqmIFiHQVAiVOHcGICSIU2WhABPQeSOeU0Nn4fov/o xIfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=5TmZ3GA7LvmXHit8kAa24HeUaVa3IsIu64SOGOdKiTk=; b=UpkT+iTye7vM8yHzQ0kusW4ksaNz1MQc/ABTsHav7r/0M8iwIYyFsBRmTYJuQ0P2AH eld7ZgMXs5ndiAuFws5kFw8nd0gh+l9pTiwpIcKduBcy4rK9atTDjbuETygzeYbXYAxt jxbhht8M302qoMadaFotG1C6O3mggvxP3pT5PnYwFda1CoLaR4OiekkPdJ70Gi/H3djs TxNQ5Wh66QADUEhE9w+qyCHULZTCdvxAQ5aM7hGgf+LMYJyP/9gXAaaTMZVOOFg8ZgiQ yoyWFJhpko7Oq84WTZeIcth3rXK7oYPXFIFfA1sLHJkXkLlklkimsqKbrXeD3yhrDFAB kHYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BGz3ciIA; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 3-v6si6412081pll.585.2018.02.26.00.22.38; Mon, 26 Feb 2018 00:22:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=BGz3ciIA; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752438AbeBZIWe (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:34 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:38007 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751879AbeBZIW2 (ORCPT ); Mon, 26 Feb 2018 03:22:28 -0500 Received: by mail-pl0-f68.google.com with SMTP id d4so8862105pll.5 for ; Mon, 26 Feb 2018 00:22:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5TmZ3GA7LvmXHit8kAa24HeUaVa3IsIu64SOGOdKiTk=; b=BGz3ciIAPa6Rjelr+9iL940YpPlOAsc88Wussw5yVjLzmuLgU5wr4efrblTqRCjsPV PzNQw7WYngn2jlG0GbIK6qOwhpUQH5b5EzpHvi86R7M8pTfgOVxqz/3OAXZxkE3LBYtm fB0a3SluWGJ0CRuF4QIzUCN7nYpet6pQM3Wm0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5TmZ3GA7LvmXHit8kAa24HeUaVa3IsIu64SOGOdKiTk=; b=gTAJmWoRp+WT6Qa+e5Qkt6KIZOHktXGS59Ve3owvOf3GCHQ5lgb9t1PNAPiNM4o4n3 TNpDmN7ZxtTfxUEY1VoBgQjgajprHtNSVR77/H7zPGaf3B8W9zqZToYx/3QAuYnG8tfN yAZooIwMxl2VVDdbDqfEmpxQbtTeDxLKOfT2Xx2mwItYC2ZSoui7922YtJXvCVJQz6HN vsJqSq1qvk8aK7j8gyaeehAabls4EbfuXgDU6vmquW5yvmTtAdioOhAdvDd5dKbQ1PS8 kB0vR5wGQqMOQmxXgf/v/Erk1CAmlKq+/IuWfDGDJ8HuB8m1wzxr6VCvQaYlmqdWhiow R/hg== X-Gm-Message-State: APf1xPACaNhrxxTVq+r9HgCefEl3Er1RT2Fy9B0qTgeCG/8cgea7cZS5 dIAksOZDkUOk0Dmo6RKjnhN+Ew== X-Received: by 2002:a17:902:ab8f:: with SMTP id f15-v6mr9909729plr.214.1519633347730; Mon, 26 Feb 2018 00:22:27 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:27 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: Dave Martin Subject: [PATCH 11/52] arm64: syscallno is secretly an int, make it official Date: Mon, 26 Feb 2018 16:19:45 +0800 Message-Id: <1519633227-29832-12-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Dave Martin commit 35d0e6fb4d upstream. The upper 32 bits of the syscallno field in thread_struct are handled inconsistently, being sometimes zero extended and sometimes sign-extended. In fact, only the lower 32 bits seem to have any real significance for the behaviour of the code: it's been OK to handle the upper bits inconsistently because they don't matter. Currently, the only place I can find where those bits are significant is in calling trace_sys_enter(), which may be unintentional: for example, if a compat tracer attempts to cancel a syscall by passing -1 to (COMPAT_)PTRACE_SET_SYSCALL at the syscall-enter-stop, it will be traced as syscall 4294967295 rather than -1 as might be expected (and as occurs for a native tracer doing the same thing). Elsewhere, reads of syscallno cast it to an int or truncate it. There's also a conspicuous amount of code and casting to bodge around the fact that although semantically an int, syscallno is stored as a u64. Let's not pretend any more. In order to preserve the stp x instruction that stores the syscall number in entry.S, this patch special-cases the layout of struct pt_regs for big endian so that the newly 32-bit syscallno field maps onto the low bits of the stored value. This is not beautiful, but benchmarking of the getpid syscall on Juno suggests indicates a minor slowdown if the stp is split into an stp x and stp w. Signed-off-by: Dave Martin Acked-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/processor.h | 2 +- arch/arm64/include/asm/ptrace.h | 9 ++++++++- arch/arm64/kernel/entry.S | 34 +++++++++++++++++----------------- arch/arm64/kernel/ptrace.c | 2 +- arch/arm64/kernel/signal.c | 6 +++--- arch/arm64/kernel/signal32.c | 2 +- arch/arm64/kernel/traps.c | 2 +- 7 files changed, 32 insertions(+), 25 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h index 5917147..f5cdda6 100644 --- a/arch/arm64/include/asm/processor.h +++ b/arch/arm64/include/asm/processor.h @@ -130,7 +130,7 @@ struct thread_struct { static inline void start_thread_common(struct pt_regs *regs, unsigned long pc) { memset(regs, 0, sizeof(*regs)); - regs->syscallno = ~0UL; + regs->syscallno = ~0; regs->pc = pc; } diff --git a/arch/arm64/include/asm/ptrace.h b/arch/arm64/include/asm/ptrace.h index ada08b5..7721d7a 100644 --- a/arch/arm64/include/asm/ptrace.h +++ b/arch/arm64/include/asm/ptrace.h @@ -116,7 +116,14 @@ struct pt_regs { }; }; u64 orig_x0; - u64 syscallno; +#ifdef __AARCH64EB__ + u32 unused2; + s32 syscallno; +#else + s32 syscallno; + u32 unused2; +#endif + u64 orig_addr_limit; u64 unused; // maintain 16 byte alignment }; diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 6915697..48c41ff 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -116,8 +116,8 @@ * Set syscallno to -1 by default (overridden later if real syscall). */ .if \el == 0 - mvn x21, xzr - str x21, [sp, #S_SYSCALLNO] + mvn w21, wzr + str w21, [sp, #S_SYSCALLNO] .endif /* @@ -232,8 +232,9 @@ alternative_else_nop_endif * * x7 is reserved for the system call number in 32-bit mode. */ -sc_nr .req x25 // number of system calls -scno .req x26 // syscall number +wsc_nr .req w25 // number of system calls +wscno .req w26 // syscall number +xscno .req x26 // syscall number (zero-extended) stbl .req x27 // syscall table pointer tsk .req x28 // current thread_info @@ -519,8 +520,8 @@ el0_svc_compat: * AArch32 syscall handling */ adrp stbl, compat_sys_call_table // load compat syscall table pointer - uxtw scno, w7 // syscall number in w7 (r7) - mov sc_nr, #__NR_compat_syscalls + mov wscno, w7 // syscall number in w7 (r7) + mov wsc_nr, #__NR_compat_syscalls b el0_svc_naked .align 6 @@ -741,19 +742,19 @@ ENDPROC(ret_from_fork) .align 6 el0_svc: adrp stbl, sys_call_table // load syscall table pointer - uxtw scno, w8 // syscall number in w8 - mov sc_nr, #__NR_syscalls + mov wscno, w8 // syscall number in w8 + mov wsc_nr, #__NR_syscalls el0_svc_naked: // compat entry point - stp x0, scno, [sp, #S_ORIG_X0] // save the original x0 and syscall number + stp x0, xscno, [sp, #S_ORIG_X0] // save the original x0 and syscall number enable_dbg_and_irq ct_user_exit 1 ldr x16, [tsk, #TI_FLAGS] // check for syscall hooks tst x16, #_TIF_SYSCALL_WORK b.ne __sys_trace - cmp scno, sc_nr // check upper syscall limit + cmp wscno, wsc_nr // check upper syscall limit b.hs ni_sys - ldr x16, [stbl, scno, lsl #3] // address in the syscall table + ldr x16, [stbl, xscno, lsl #3] // address in the syscall table blr x16 // call sys_* routine b ret_fast_syscall ni_sys: @@ -767,24 +768,23 @@ ENDPROC(el0_svc) * switches, and waiting for our parent to respond. */ __sys_trace: - mov w0, #-1 // set default errno for - cmp scno, x0 // user-issued syscall(-1) + cmp wscno, #-1 // user-issued syscall(-1)? b.ne 1f - mov x0, #-ENOSYS + mov x0, #-ENOSYS // set default errno if so str x0, [sp, #S_X0] 1: mov x0, sp bl syscall_trace_enter cmp w0, #-1 // skip the syscall? b.eq __sys_trace_return_skipped - uxtw scno, w0 // syscall number (possibly new) + mov wscno, w0 // syscall number (possibly new) mov x1, sp // pointer to regs - cmp scno, sc_nr // check upper syscall limit + cmp wscno, wsc_nr // check upper syscall limit b.hs __ni_sys_trace ldp x0, x1, [sp] // restore the syscall args ldp x2, x3, [sp, #S_X2] ldp x4, x5, [sp, #S_X4] ldp x6, x7, [sp, #S_X6] - ldr x16, [stbl, scno, lsl #3] // address in the syscall table + ldr x16, [stbl, xscno, lsl #3] // address in the syscall table blr x16 // call sys_* routine __sys_trace_return: diff --git a/arch/arm64/kernel/ptrace.c b/arch/arm64/kernel/ptrace.c index 8eedeef..193c621 100644 --- a/arch/arm64/kernel/ptrace.c +++ b/arch/arm64/kernel/ptrace.c @@ -1346,7 +1346,7 @@ static void tracehook_report_syscall(struct pt_regs *regs, if (dir == PTRACE_SYSCALL_EXIT) tracehook_report_syscall_exit(regs, 0); else if (tracehook_report_syscall_entry(regs)) - regs->syscallno = ~0UL; + regs->syscallno = ~0; regs->regs[regno] = saved_reg; } diff --git a/arch/arm64/kernel/signal.c b/arch/arm64/kernel/signal.c index 404dd67..fd7eba8 100644 --- a/arch/arm64/kernel/signal.c +++ b/arch/arm64/kernel/signal.c @@ -113,7 +113,7 @@ static int restore_sigframe(struct pt_regs *regs, /* * Avoid sys_rt_sigreturn() restarting. */ - regs->syscallno = ~0UL; + regs->syscallno = ~0; err |= !valid_user_regs(®s->user_regs, current); @@ -332,7 +332,7 @@ static void do_signal(struct pt_regs *regs) { unsigned long continue_addr = 0, restart_addr = 0; int retval = 0; - int syscall = (int)regs->syscallno; + int syscall = regs->syscallno; struct ksignal ksig; /* @@ -346,7 +346,7 @@ static void do_signal(struct pt_regs *regs) /* * Avoid additional syscall restarting via ret_to_user. */ - regs->syscallno = ~0UL; + regs->syscallno = ~0; /* * Prepare for system call restart. We do this here so that a diff --git a/arch/arm64/kernel/signal32.c b/arch/arm64/kernel/signal32.c index b7063de..1effea2 100644 --- a/arch/arm64/kernel/signal32.c +++ b/arch/arm64/kernel/signal32.c @@ -354,7 +354,7 @@ static int compat_restore_sigframe(struct pt_regs *regs, /* * Avoid compat_sys_sigreturn() restarting. */ - regs->syscallno = ~0UL; + regs->syscallno = ~0; err |= !valid_user_regs(®s->user_regs, current); diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index c743d1f..59f80b0 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -541,7 +541,7 @@ asmlinkage long do_ni_syscall(struct pt_regs *regs) if (show_unhandled_signals_ratelimited()) { pr_info("%s[%d]: syscall %d\n", current->comm, - task_pid_nr(current), (int)regs->syscallno); + task_pid_nr(current), regs->syscallno); dump_instr("", regs); if (user_mode(regs)) __show_regs(regs); From patchwork Mon Feb 26 08:19:46 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129568 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356356lja; Mon, 26 Feb 2018 00:22:47 -0800 (PST) X-Google-Smtp-Source: AH8x225qRR08iFX/3kveFtHFfm3BwyP8aEqceeIditANB6Ttpp7NuCu2Ct4rrJ2Qz9QoB1UxGbCE X-Received: by 10.98.103.136 with SMTP id t8mr9832553pfj.177.1519633367063; Mon, 26 Feb 2018 00:22:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633367; cv=none; d=google.com; s=arc-20160816; b=dTu2cRjtYJlb6ijbDwFc83sZ9fz7HE1fGQpRRHwKsIGyQe3JT3L8+8ouD3LNEhNKzg O6OoA+Zq5WXRpoizfa9vGoE8hWCLuRYBpUSZPQE0IjiJbH+FkZdn+g/ncXRiKzh3NatT JiAo2agMsJ4WvVoByswNlnQzNmu39XysCD8JOJlNd56wBDAflRhUfr3LzoJ6OXU0r+Rv e09/RAm58ljZp4n64xYVnLIv9mBBGazy5eXVRoNRTomWBE9DFHg82ebSHSdsUx9Fk3s3 W6QgYM+/C1cjb2xoMWJ9MIy504MkQHUxOkPykj5kysBaDauF8EIDys+EYQIn80kUI7z8 XMUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=taE8mK2Ip25XXu3tH/TfrJ9TiKkrcCk7pgR93VQ9+cc=; b=mDB1JR13H8ejA72ceZQ5cr1EsycXXugT4ex+7XaISx534ltuSYP5Q2IRICNcm6O2nw XDp2HRuWIdhV4Sz6y2suYzq5D+Hz3KsI1bNQBXjtnwiqOhjlrxSZgrNNrrplkiecOfYL JyU0Z3K326vbQIY20yddASb3hzkcRP/UAJa2PI8Zzr8yb1UbDiFILnvAq44r1oKJvXuK 0wrx7ZQEriALRZ3/Ag8YeezQLIm1IN2aB3M1RxGUH4Ig6SH9+RdYb8cEi4zvmsSbsBLy IytUVy0BYKJEWCFAXYFgzOX+q18BQY30Jy/8R57Pt+rR5m2Q6n8omCLY8grsDIK7nY5V 3K3g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MDioq7zh; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m22si5224509pgv.643.2018.02.26.00.22.46; Mon, 26 Feb 2018 00:22:47 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MDioq7zh; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752447AbeBZIWm (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:42 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:45458 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751932AbeBZIWf (ORCPT ); Mon, 26 Feb 2018 03:22:35 -0500 Received: by mail-pf0-f194.google.com with SMTP id j24so6216212pff.12 for ; Mon, 26 Feb 2018 00:22:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=taE8mK2Ip25XXu3tH/TfrJ9TiKkrcCk7pgR93VQ9+cc=; b=MDioq7zh8inaTCPNZ072hhyYNNWki2OkWhxbwQ6dM+71A0eOxwifTmr1PNxSsMlCiB 28eHc0HPvTSAnZ3GOqiw9WtcGyugYYaGl8wORxByDQstGgjdV+17xi6wZMw8XtKBtPtw vaC1IdStpPgCtiKChJvYegndbU7nTrUhWU6Qk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=taE8mK2Ip25XXu3tH/TfrJ9TiKkrcCk7pgR93VQ9+cc=; b=El9EUq4BsvT5V0oV19tdstpa7vY3K0TnQvOnRhqaXo5GaVhaHpa2qjW51PNr0NlddD b/f4p2dH5lbZDd3cMh6jmMaT9e3aDTEfEqa0Fqc/3wpfmz1xondmIqydsDg2S7GvyHYd Mh6IUGuw4mHJCndFer0Aw+dX0/rE4tuvb9duXSi6yTFRYK7DcOj+W4UoMfFYP5PLe1eP b4zXmZaktDd2cmqLIPrU4LzQvbeImXXANxNQTB4rffyDUazuUrDUUzEiA8/Ir5y6pw0q aNshhToa4kFxCw38af5BsioLxf1r1sqBD1pRu57ecV435kPUwWjd+iuAl59irBTFJiFR JNfw== X-Gm-Message-State: APf1xPADdTCHTM1LI8s3Evu1JzyD7p30lYx20g9DOBXCErXV+CdT9aVs XfBh4u+NL8smGVwVQbfqToAVyg== X-Received: by 10.98.245.131 with SMTP id b3mr9842362pfm.20.1519633355077; Mon, 26 Feb 2018 00:22:35 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:34 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 12/52] arm64: entry: Ensure branch through syscall table is bounded under speculation Date: Mon, 26 Feb 2018 16:19:46 +0800 Message-Id: <1519633227-29832-13-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 6314d90e6493 upstream. In a similar manner to array_index_mask_nospec, this patch introduces an assembly macro (mask_nospec64) which can be used to bound a value under speculation. This macro is then used to ensure that the indirect branch through the syscall table is bounded under speculation, with out-of-range addresses speculating as calls to sys_io_setup (0). Reviewed-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/assembler.h | 11 +++++++++++ arch/arm64/kernel/entry.S | 2 ++ 2 files changed, 13 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index ab3a179..a109d30 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -94,6 +94,17 @@ .endm /* + * Sanitise a 64-bit bounded index wrt speculation, returning zero if out + * of bounds. + */ + .macro mask_nospec64, idx, limit, tmp + sub \tmp, \idx, \limit + bic \tmp, \tmp, \idx + and \idx, \idx, \tmp, asr #63 + csdb + .endm + +/* * NOP sequence */ .macro nops, num diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 48c41ff..0a27e12 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -233,6 +233,7 @@ alternative_else_nop_endif * x7 is reserved for the system call number in 32-bit mode. */ wsc_nr .req w25 // number of system calls +xsc_nr .req x25 // number of system calls (zero-extended) wscno .req w26 // syscall number xscno .req x26 // syscall number (zero-extended) stbl .req x27 // syscall table pointer @@ -754,6 +755,7 @@ el0_svc_naked: // compat entry point b.ne __sys_trace cmp wscno, wsc_nr // check upper syscall limit b.hs ni_sys + mask_nospec64 xscno, xsc_nr, x19 // enforce bounds for syscall number ldr x16, [stbl, xscno, lsl #3] // address in the syscall table blr x16 // call sys_* routine b ret_fast_syscall From patchwork Mon Feb 26 08:19:47 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129569 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356432lja; Mon, 26 Feb 2018 00:22:53 -0800 (PST) X-Google-Smtp-Source: AH8x226MA7FR6kAML62tKfIzqFyGzy943e+ZWpN9xhyIM9w/XK5VLdSlMumz2GMQXF6E6hc3c6Ty X-Received: by 2002:a17:902:d20a:: with SMTP id t10-v6mr9913774ply.257.1519633373308; Mon, 26 Feb 2018 00:22:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633373; cv=none; d=google.com; s=arc-20160816; b=Ok/0+CJA85Y5UThV/beZGAOt/ALr/tzKx63AiAQ3MD2TWe+FJgRZ537t2Ku/lz7Jjq j3KenhMzKmKfET3lxAW2BO4nTqab4KHJ2In26jwEtVugDcS4ob/AEvzaR0fRcvLnPaw0 HIgwaRbEny5VlSuYBrHtjmdg/37lWN/TdurZVcuaxUW/jl/UdJZ/nMuWnBKiBa9UD9ZN 4mG/eJnzsQLj9RPeQpK9nZSVxrulzs0f4A+nsj5gSDb/ISgZbj7AaeBdDvQAQizxIoQ+ T+eFhtvj3CtAdxIT53zdmPuYb9OX4hDoztroxYYomIr7KA3pL3QZmFC7DDqyAhj6Lwn8 mRfQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=XoltgftvsVb6cg5EoqmIT2rqheB1dI0xAlZ6dp7AzjE=; b=MikrICAlhvKbMJW+iQeDutfBIOsr4ZLTqhfkKvemNUic0Qwqsb+7p8hpijkaPrB6ZJ orDEQOrSJuF2wyt+5aGiVaLc8cBakFyeRAoUMQ2Tl6KmLnG5lz2ewHOFJ5bnq71hJoTw Tg+WHKvcs6wHLuGpsjTpOBuMamqRENWoBYJM7fOVThypreyIZTuToG+2JSaOEXF3BRcX FUjJ4pE0shpXaVhRheiloyy9NssTB297pf79cpwpSVETXjx4+dIqzyusF0zMCXNKiMGL DnTOH0Z62wKjMe9oJACdb04CfOb8y6+hzT3gnLcKqEDa9LgxS7zt4Uq3MoWdSxM7A3t/ p90g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gwp5XxQ5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c6si5261838pgt.111.2018.02.26.00.22.52; Mon, 26 Feb 2018 00:22:53 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gwp5XxQ5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752458AbeBZIWt (ORCPT + 28 others); Mon, 26 Feb 2018 03:22:49 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:42877 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752449AbeBZIWn (ORCPT ); Mon, 26 Feb 2018 03:22:43 -0500 Received: by mail-pg0-f66.google.com with SMTP id y8so5905259pgr.9 for ; Mon, 26 Feb 2018 00:22:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=XoltgftvsVb6cg5EoqmIT2rqheB1dI0xAlZ6dp7AzjE=; b=Gwp5XxQ5/4x2yaA+PNjtbUdklQAqRcBSTLsVH2yEcJ82PNXHXaU0l1ySt5nIRcTuGK 4R8cZTxwdd6iUCur12qIqRiJc0U+vm8Jf79x2yclDajkrNKApkZ5eqzGc2z55Hev20Vq n6jl9CmDbpIxQFDB6z2DjMxnVnJbEOp22qwQo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=XoltgftvsVb6cg5EoqmIT2rqheB1dI0xAlZ6dp7AzjE=; b=EujGS2/mW5dboMXU2CeUxSzz90ADYlgqHuKmNu9j/Gwe3FceAIGEEDNVt38dWUyOAk vpm6od7VG1E6llI2p9VhaJc3ZwusZWadK1TgyR3o187Dv0D4qZDlkyD+Hwl/S0txfuxQ 50+Bcff2di6qor8N2f3A7NYAfSgeMMntKjzysSE0EwDtzV4DgqYkgZ4E56by/vJZq0f4 yXAmpie46tlZ1+fOwbiTgIcui/d5Jbf1zOWEE8xvaC+i9dIhsZZFXlSDMwVkkrZ+xqY/ kB/Qmi6bFznzX8p4GRSPtLs81WUu6MKPgqvsqamB3tIIsfS9tIr7+mQhmonFQRiJLpLg qU1g== X-Gm-Message-State: APf1xPBCBtvmttTOiumBvnuwqGc5fNLIJ/rOSFtDsffQZt2BM1eeoMaU +WwX1pFdMln0ORZ5Nhm1PP6Wig== X-Received: by 10.99.5.197 with SMTP id 188mr7795225pgf.196.1519633362563; Mon, 26 Feb 2018 00:22:42 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.35 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:42 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 13/52] arm64: uaccess: Prevent speculative use of the current addr_limit Date: Mon, 26 Feb 2018 16:19:47 +0800 Message-Id: <1519633227-29832-14-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit c2f0ad4fc089 upstream. A mispredicted conditional call to set_fs could result in the wrong addr_limit being forwarded under speculation to a subsequent access_ok check, potentially forming part of a spectre-v1 attack using uaccess routines. This patch prevents this forwarding from taking place, but putting heavy barriers in set_fs after writing the addr_limit. Reviewed-by: Mark Rutland Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: no set_thread_flag(TIF_FSCHECK) in arch/arm64/include/asm/uaccess.h --- arch/arm64/include/asm/uaccess.h | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 3531fec..00025c5 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -71,6 +71,13 @@ static inline void set_fs(mm_segment_t fs) current_thread_info()->addr_limit = fs; /* + * Prevent a mispredicted conditional call to set_fs from forwarding + * the wrong address limit to access_ok under speculation. + */ + dsb(nsh); + isb(); + + /* * Enable/disable UAO so that copy_to_user() etc can access * kernel memory with the unprivileged instructions. */ From patchwork Mon Feb 26 08:19:48 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129570 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356573lja; Mon, 26 Feb 2018 00:23:05 -0800 (PST) X-Google-Smtp-Source: AH8x226bxuXiZmGuGlFLkDOJPRU67Upm5MNxkS7Bb3MlcHP1cARNrGdcT+oA9FzBZXLhRLIeTy0J X-Received: by 10.98.225.2 with SMTP id q2mr9709344pfh.23.1519633385264; Mon, 26 Feb 2018 00:23:05 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633385; cv=none; d=google.com; s=arc-20160816; b=ypQonxn1zd/ntshrCS/LVEdcVQ2XAfYlaa6bGB+m/zxE2zr8Jt1EVyb4xAiBAXfCK7 5WB660urEbe9aDCrpY/Mr2jvYguLOuH9U/cjhL+o4FzouRQVBkqvYmByeI1aUQ+1F9Fl sVhruv30IHRssf0IRxMBmgY4uQEuyVn8dAYykG5kYakv6kre/dywiTBB542L6BPF953H nyaGUdhzL8DPQNZlTwlfsDB7W4Le9/xPxFjK49SEEO3gsex1DJ/nTjSb09Uwj6ws+0In OsyP7ckpR7lROLg2f1/pd+f65Obpp23uI7tvBqZYvw+/Rz/wBCmzgoUrHGHpO4tlE8K2 lV7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=JeYitV85vYUZHZtgTIJwAzhhdYjQpmFJh73a4N4z6FM=; b=KAhg1LG9txbSU1MVQmdtmqD6ke1kAiC8O8o+qRjXxzEfaCOo3sQHab0zxz2f0yibCl qWpYp9fGAFjRpDtotlYWOQ3n2Hvx3hJxZGpdqtYambUAsy5UpbycUY/1NwO8fR7bD6x8 gIdq4nXnTjkdDqZ6mLrv6l21xyn15ln6mHsngmKhhLjgodNo8x3bEAVtR5YVHAe94+Cg CtcXQtLhShOlxBAhTVNoLzAo2DwGOwEbJQioC1q0eV6zQw2TqFwgrBrU4i8MW5KfW4hb LuNgcSgfSVsbuR1JSZPeyeepWtTieOY3/x++uPXxx7oyJKjzoqlmddnGbuLShpurPhiD 8ozQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WuEnGHaW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t84si5267180pgb.241.2018.02.26.00.23.04; Mon, 26 Feb 2018 00:23:05 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WuEnGHaW; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752485AbeBZIXB (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:01 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:38034 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752460AbeBZIWv (ORCPT ); Mon, 26 Feb 2018 03:22:51 -0500 Received: by mail-pl0-f68.google.com with SMTP id d4so8862490pll.5 for ; Mon, 26 Feb 2018 00:22:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=JeYitV85vYUZHZtgTIJwAzhhdYjQpmFJh73a4N4z6FM=; b=WuEnGHaWFEDfKx3hNgfDxwVUV612UXDp7O3vkuvzzYqTTTrRDssZ4a0LQ5AbaWbII2 hzmuQlibDWAxkE6coVIxiK06vBbRzCmkQ2mfCW0sEhnlQG+lR4WUzlEmRlj1dMSBCngG q5Sd4quYSTPM2z5cEOKOj/QcT793ca0IgWZtQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=JeYitV85vYUZHZtgTIJwAzhhdYjQpmFJh73a4N4z6FM=; b=lvdT2/DSrFnfTpSKBAi2M/qDxxBLMSBSgOOwsMbduAMa3Cd4lgsWmy4l5bSqbDzTod z098iuJJ5zI67RF+6YXC46nuVXROwDy9DybFRh75FNRCr/u8ZNcZ2SoEkqvl14bEX+0n WQ7vrJpYZhs4nGKH0DBOjk1goPS4YhsfEmi3KE8MJ1USnJNN9Gjt5uxWFbCEe6Nhq2Jb sj0SiMcVCFaSxP+rIf4y49aY4LIBBl6Fa90zd63M/KP+fuhSKxQjm/F6+4wRrnwoDVKM p+G4+96PkBlQ7J4TvGR6KyPZNtaBMs68ihEAQW0iAuqLPGTWl1Z+EhBnh9OF4JYrtEHZ gnQA== X-Gm-Message-State: APf1xPAr2bXfcVGeLH9f5o+QDgxNd8PAAbqPoOtIF9VHYWKA/5HjcJ7r IgNbTnflPZgzFU1Xq+aztCcLQA== X-Received: by 2002:a17:902:34f:: with SMTP id 73-v6mr9561276pld.55.1519633370036; Mon, 26 Feb 2018 00:22:50 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:49 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 14/52] arm64: uaccess: Don't bother eliding access_ok checks in __{get, put}_user Date: Mon, 26 Feb 2018 16:19:48 +0800 Message-Id: <1519633227-29832-15-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 84624087dd7e upstream. access_ok isn't an expensive operation once the addr_limit for the current thread has been loaded into the cache. Given that the initial access_ok check preceding a sequence of __{get,put}_user operations will take the brunt of the miss, we can make the __* variants identical to the full-fat versions, which brings with it the benefits of address masking. The likely cost in these sequences will be from toggling PAN/UAO, which we can address later by implementing the *_unsafe versions. Reviewed-by: Robin Murphy Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: keep __{get/put}_user_unaligned in arch/arm64/include/asm/uaccess.h --- arch/arm64/include/asm/uaccess.h | 54 ++++++++++++++++++++++++---------------- 1 file changed, 32 insertions(+), 22 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index 00025c5..ffa4e39 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -248,30 +248,35 @@ do { \ (x) = (__force __typeof__(*(ptr)))__gu_val; \ } while (0) -#define __get_user(x, ptr) \ +#define __get_user_check(x, ptr, err) \ ({ \ - int __gu_err = 0; \ - __get_user_err((x), (ptr), __gu_err); \ - __gu_err; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_READ, __p, sizeof(*__p))) { \ + __p = uaccess_mask_ptr(__p); \ + __get_user_err((x), __p, (err)); \ + } else { \ + (x) = 0; (err) = -EFAULT; \ + } \ }) #define __get_user_error(x, ptr, err) \ ({ \ - __get_user_err((x), (ptr), (err)); \ + __get_user_check((x), (ptr), (err)); \ (void)0; \ }) #define __get_user_unaligned __get_user -#define get_user(x, ptr) \ +#define __get_user(x, ptr) \ ({ \ - __typeof__(*(ptr)) __user *__p = (ptr); \ - might_fault(); \ - access_ok(VERIFY_READ, __p, sizeof(*__p)) ? \ - __p = uaccess_mask_ptr(__p), __get_user((x), __p) : \ - ((x) = 0, -EFAULT); \ + int __gu_err = 0; \ + __get_user_check((x), (ptr), __gu_err); \ + __gu_err; \ }) +#define get_user __get_user + #define __put_user_asm(instr, alt_instr, reg, x, addr, err, feature) \ asm volatile( \ "1:"ALTERNATIVE(instr " " reg "1, [%2]\n", \ @@ -314,30 +319,35 @@ do { \ uaccess_disable_not_uao(); \ } while (0) -#define __put_user(x, ptr) \ +#define __put_user_check(x, ptr, err) \ ({ \ - int __pu_err = 0; \ - __put_user_err((x), (ptr), __pu_err); \ - __pu_err; \ + __typeof__(*(ptr)) __user *__p = (ptr); \ + might_fault(); \ + if (access_ok(VERIFY_WRITE, __p, sizeof(*__p))) { \ + __p = uaccess_mask_ptr(__p); \ + __put_user_err((x), __p, (err)); \ + } else { \ + (err) = -EFAULT; \ + } \ }) #define __put_user_error(x, ptr, err) \ ({ \ - __put_user_err((x), (ptr), (err)); \ + __put_user_check((x), (ptr), (err)); \ (void)0; \ }) #define __put_user_unaligned __put_user -#define put_user(x, ptr) \ +#define __put_user(x, ptr) \ ({ \ - __typeof__(*(ptr)) __user *__p = (ptr); \ - might_fault(); \ - access_ok(VERIFY_WRITE, __p, sizeof(*__p)) ? \ - __p = uaccess_mask_ptr(__p), __put_user((x), __p) : \ - -EFAULT; \ + int __pu_err = 0; \ + __put_user_check((x), (ptr), __pu_err); \ + __pu_err; \ }) +#define put_user __put_user + extern unsigned long __must_check __arch_copy_from_user(void *to, const void __user *from, unsigned long n); extern unsigned long __must_check __arch_copy_to_user(void __user *to, const void *from, unsigned long n); extern unsigned long __must_check __copy_in_user(void __user *to, const void __user *from, unsigned long n); From patchwork Mon Feb 26 08:19:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129612 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3366221lja; Mon, 26 Feb 2018 00:36:46 -0800 (PST) X-Google-Smtp-Source: AH8x2242SPIjaLsJ2dQB4A9Woovd10/D19L9Sfknra0AuwhPweQClnoywpnbDaMod3KzEGs+d0Mq X-Received: by 10.101.87.132 with SMTP id b4mr7698291pgr.282.1519634206587; Mon, 26 Feb 2018 00:36:46 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519634206; cv=none; d=google.com; s=arc-20160816; b=CFmZDz56zHd+aeJSTi4LKNDx7gPRAcr0A4iO5bJlle9jPsJtU5cy5zqwJf9WPiMffp 8C9xdTmfoMkP4ELkzG+Wv2fSSBXjbiBeX2Q0Lhre18AXgxn1QETne6ONp4vNPDEEO0RI 408/WSg2ObRAsZKVMaM1YcbGqOjIjB2Dksq6wuT8VoYZ+gmLczJ8X6ta0pN6Zri0AXD+ +QRwcSCan6uN+vgbuiNKvisDDw8YSPABN1TRJpXjPb8Oo5dnHuhhBac+Ime5AJVRXbln NizPPlHLqx/OaIrnwH6J84nGWZ85qyWlEcUCNReV1J1zkThP19PKyag8Nv4OWKOWrq+i UTHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=iAPTv5jlMzw6ovOzicmoLxGXCxHRzs8GBzIryr4d66U=; b=V8rSjfDirA6JqxPs6Rj0tSd1Fmd7DDG2fZS4XtfVfMc+Vq9S2cZdPXNPgofqszneNT Z26PN5njMhMtlK4wVXSby2W8I6stkhO1FA5MINbkACjxcGo2rpSHhioOcGnfsepKRQZt ozKjdzUlb6gJw+T6Tg4mrbGk8ZqF0LS4ndPjvBGYlk5R6pt8re3/G0MXd6FD7aiSvgDo alzoQeU+5I1SZ4Yu0sFOfF61FRiXeo331HmSU8g3joqGGluG9Pk+30kl9OeffIo4hRBX ZWLivQEwpbf5tSrC8wZ8hx7EgGroTyt3lSxmha2P1zMauyyDqK91ZQlnEq82Z/NnrywJ XSRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YBfQoz1u; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id g9si5238858pgs.752.2018.02.26.00.36.46; Mon, 26 Feb 2018 00:36:46 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YBfQoz1u; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752500AbeBZIgl (ORCPT + 28 others); Mon, 26 Feb 2018 03:36:41 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:46556 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752037AbeBZIW5 (ORCPT ); Mon, 26 Feb 2018 03:22:57 -0500 Received: by mail-pg0-f66.google.com with SMTP id r26so676279pgv.13 for ; Mon, 26 Feb 2018 00:22:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=iAPTv5jlMzw6ovOzicmoLxGXCxHRzs8GBzIryr4d66U=; b=YBfQoz1u8brYY/frdZ/VdSuZzj7DLeGfREeI7i8Y/sPfBRTyFDQkAYxMUYrpw5GqyE q/eRL1rg0vsXX4DYVO0U0sS0hXDilH7xHSOtyHiWOMZvU/MdVm2yzd8DL9U5u2hA6/cC wD0iDLuVRmtMUiXpjvyxmzBZNSeq/tOrGj+fY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=iAPTv5jlMzw6ovOzicmoLxGXCxHRzs8GBzIryr4d66U=; b=lt7eraSC3/an94+nky1b6mlNvkpOEVE7aIAaBBm6ZvgkgkAXjxza2IjHuy5VJm4s2u gUvWKomoCKPiCo0FDfGzT6OdfwXKTPIpbjCP/sfNaELyLf60tIAADcnkatH0d9HEadi0 XK+SU87GNVXaTHAOgVYr9tYIPez9zKaj75196WU5wgJR8azGQIKnRKPNaAjKGJi6C5sx kW+w7ydOgXxpc1wKJbKDHvvW68BZHYWTFdGgcmjn0KIG7FeU1FvJcr6Zi2Thp36QfsyR 7CKXNhHh4sWR4uMsJcUuYFgvh2F6DzqsOkYsDrHxs2H7Au/Gpi4OFbjIOrjEPm3kTljY MxIA== X-Gm-Message-State: APf1xPB2GUhfxCCfUjbYEs0675gfFhcpUWGCjDEH1z9qZfZ2I7tjJC1g DggZ0sl5YlQ9bMW5ElbtSxSMbg== X-Received: by 10.99.125.92 with SMTP id m28mr7923145pgn.257.1519633377354; Mon, 26 Feb 2018 00:22:57 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:22:56 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 15/52] arm64: uaccess: Mask __user pointers for __arch_{clear, copy_*}_user Date: Mon, 26 Feb 2018 16:19:49 +0800 Message-Id: <1519633227-29832-16-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon Rewritting from commit f71c2ffcb20d upstream. On LTS 4.9, there has no raw_copy_from/to_user, neither __copy_user_flushcache, and it isn't good idead to pick up them. The following is origin commit log, that's also applicable for the new patch. Like we've done for get_user and put_user, ensure that user pointers are masked before invoking the underlying __arch_{clear,copy_*}_user operations. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/include/asm/uaccess.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uaccess.h index ffa4e39..fbf4ce4 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -357,14 +357,14 @@ static inline unsigned long __must_check __copy_from_user(void *to, const void _ { kasan_check_write(to, n); check_object_size(to, n, false); - return __arch_copy_from_user(to, from, n); + return __arch_copy_from_user(to, __uaccess_mask_ptr(from), n); } static inline unsigned long __must_check __copy_to_user(void __user *to, const void *from, unsigned long n) { kasan_check_read(from, n); check_object_size(from, n, true); - return __arch_copy_to_user(to, from, n); + return __arch_copy_to_user(__uaccess_mask_ptr(to), from, n); } static inline unsigned long __must_check copy_from_user(void *to, const void __user *from, unsigned long n) @@ -374,7 +374,7 @@ static inline unsigned long __must_check copy_from_user(void *to, const void __u check_object_size(to, n, false); if (access_ok(VERIFY_READ, from, n)) { - res = __arch_copy_from_user(to, from, n); + res = __arch_copy_from_user(to, __uaccess_mask_ptr(from), n); } if (unlikely(res)) memset(to + (n - res), 0, res); @@ -387,7 +387,7 @@ static inline unsigned long __must_check copy_to_user(void __user *to, const voi check_object_size(from, n, true); if (access_ok(VERIFY_WRITE, to, n)) { - n = __arch_copy_to_user(to, from, n); + n = __arch_copy_to_user(__uaccess_mask_ptr(to), from, n); } return n; } @@ -395,7 +395,7 @@ static inline unsigned long __must_check copy_to_user(void __user *to, const voi static inline unsigned long __must_check copy_in_user(void __user *to, const void __user *from, unsigned long n) { if (access_ok(VERIFY_READ, from, n) && access_ok(VERIFY_WRITE, to, n)) - n = __copy_in_user(to, from, n); + n = __copy_in_user(__uaccess_mask_ptr(to), __uaccess_mask_ptr(from), n); return n; } From patchwork Mon Feb 26 08:19:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129571 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356649lja; Mon, 26 Feb 2018 00:23:11 -0800 (PST) X-Google-Smtp-Source: AH8x227h+WU1V6c0j3jpi5e5l5V7IDzKgHyNUxnQLCqq2OvdIXkMTE/AyDdM3eXwJ41mqFpbMQJw X-Received: by 10.99.127.86 with SMTP id p22mr7788344pgn.157.1519633391563; Mon, 26 Feb 2018 00:23:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633391; cv=none; d=google.com; s=arc-20160816; b=vui47c7E8GvTTuD7f3JEk5eSyW9VxdCI0p4y4u41Hiw5o2oykTpnbRbF2iPh8V1gsA FQ/ZlxYn4vr/2JYljUHR20juODcS7kr0dLhkcdESiAIaeSBvMuWN/KFXGUKKkznT5kNK RzXipR7+Veh47QHupje7LLmK9gLBY06ciz0SgyQbroSEAuAF5Vlt83IKbz0se5Dkg5hu 3zcG3C5HuhVBKBIPMJhabzuvOqRX8Drxi1LfdKe8jj2GyG9KpW/7VCvtTyzFLig5D2VB JCbZWoPOq7E0X+cHygWCSICOZxcEag10oS1BXIU7yhtFIPxEOi0qh8NsGEHT1sNIZ9zX AvmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=ieEfJQsM4sBvXx7mXsGG38weJFE2pHNKljsnVCbD9xI=; b=w5MKtWwE9ZsUP5WUbM3PwvnE6F4NdrQdX0aZ7ik9umDOe7HkpqNyOnTU987sdTTrMn MK/aF157UiJVUa78h5g3ZzWF7uOO46XoiJ16dtncZC8ve7ZHYLFpi+CK1itTJRyw1epq 7fiiX5MhZF/t+C8UwVcx73XNdFD8tvvgPsFkSmn7su1mn0T70EgWnHa+AqCsiBn79KoM cSisdq7IAgLRhBqlVLRogBPLWaE/qJ59j/aN7VmkMi5sUXO7TZVwcyM+p4hXW9Wsp1AD RBnb7mqB59OhKqnHqp9gETNKSVZAgc10vQx0CjCEQFope+dcA86EI5lhwhDPmLWbJKkX 4GyQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WUx5z9M5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id t84si5267180pgb.241.2018.02.26.00.23.11; Mon, 26 Feb 2018 00:23:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WUx5z9M5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752494AbeBZIXI (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:08 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:46508 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751793AbeBZIXE (ORCPT ); Mon, 26 Feb 2018 03:23:04 -0500 Received: by mail-pl0-f68.google.com with SMTP id x19so8861026plr.13 for ; Mon, 26 Feb 2018 00:23:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=ieEfJQsM4sBvXx7mXsGG38weJFE2pHNKljsnVCbD9xI=; b=WUx5z9M505qUpiQcj+a/NlT1Q+F8mp4sYpex56WGyCXo9DiKptlyp5wcidmum6Bkmj 0pZbywgZ5YLWwvwSldmlgPIOoMiiGkxhtTgmbZA50OCdh7dNi+GzGYhartB4YHluLHIF avK4g4hb97ldKynm0DOh7IqBQH/FRDLcSmaF0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ieEfJQsM4sBvXx7mXsGG38weJFE2pHNKljsnVCbD9xI=; b=aRPsXDmuO7/hK8PLcOXhzc8A8ytnlVxAPmqmV/TF5LWGdNFsDLmbyiBuzM+9G7ucK/ yAwFWZ5laK01BboNtJhblPYATk0z6p/9FcPK2f//Gif2UaQq0Km3Ac34ywt5j76y8GiN Lg2ABMv/KOziQqxT9RptX7YhB9MyX38WOr2zumnjH3hLJjw2jGZSCPV1El3X7OknUpeg 0kihRrwb36c/W7pDMcwCFDmfAf6NdQPRckGDE7BXKxdOfZZNCneCXrjFhp+xnNtbwg2U 8r7mpFxS6ewTKwCLeCahpunB5TiBSaXHTIeyMP16p/I7+MU+vERSGrxWx7daQvnzgcaE r+3A== X-Gm-Message-State: APf1xPDQAx9Qe9YJbxRMR3f/y5OXgsuODlBLWx5bVB1HA8mntIbonlPv Qa9ajmwTiU8+oyUbPXlLy4rPkeEDfw8= X-Received: by 2002:a17:902:b904:: with SMTP id bf4-v6mr9693178plb.195.1519633383679; Mon, 26 Feb 2018 00:23:03 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.22.57 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:03 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 16/52] arm64: futex: Mask __user pointers prior to dereference Date: Mon, 26 Feb 2018 16:19:50 +0800 Message-Id: <1519633227-29832-17-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 91b2d3442f6a upstream. The arm64 futex code has some explicit dereferencing of user pointers where performing atomic operations in response to a futex command. This patch uses masking to limit any speculative futex operations to within the user address space. Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: change on old futex_atomic_op_inuser function instead of arch_futex_atomic_op_inuser in arch/arm64/include/asm/futex.h --- arch/arm64/include/asm/futex.h | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index 85c4a89..1943cf6 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -48,13 +48,14 @@ do { \ } while (0) static inline int -futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr) +futex_atomic_op_inuser (int encoded_op, u32 __user *_uaddr) { int op = (encoded_op >> 28) & 7; int cmp = (encoded_op >> 24) & 15; int oparg = (encoded_op << 8) >> 20; int cmparg = (encoded_op << 20) >> 20; int oldval = 0, ret, tmp; + u32 __user *uaddr = __uaccess_mask_ptr(_uaddr); if (encoded_op & (FUTEX_OP_OPARG_SHIFT << 28)) oparg = 1 << oparg; @@ -106,15 +107,17 @@ futex_atomic_op_inuser (int encoded_op, u32 __user *uaddr) } static inline int -futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *uaddr, +futex_atomic_cmpxchg_inatomic(u32 *uval, u32 __user *_uaddr, u32 oldval, u32 newval) { int ret = 0; u32 val, tmp; + u32 __user *uaddr; - if (!access_ok(VERIFY_WRITE, uaddr, sizeof(u32))) + if (!access_ok(VERIFY_WRITE, _uaddr, sizeof(u32))) return -EFAULT; + uaddr = __uaccess_mask_ptr(_uaddr); uaccess_enable(); asm volatile("// futex_atomic_cmpxchg_inatomic\n" " prfm pstl1strm, %2\n" From patchwork Mon Feb 26 08:19:51 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129608 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3364323lja; Mon, 26 Feb 2018 00:34:02 -0800 (PST) X-Google-Smtp-Source: AH8x224lG+dboP/PMIrmFllxcc6+zdRTzY/KTyeYQChYsPFwoygSW2EssnWfz2k1IOUG1k4r022h X-Received: by 2002:a17:902:d205:: with SMTP id t5-v6mr10218780ply.322.1519634042047; Mon, 26 Feb 2018 00:34:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519634042; cv=none; d=google.com; s=arc-20160816; b=s1PAt99Q7dH24j3HS+3eOtZQHrzAb/URHSOat1JnJqKTklIKljFfku6iggSjI/h3w0 lkhovbYaCNl6Vhtwf9dIrB5apNgdA5WFqqsRIrIvdTY29IqprlJ7EtMNazOUtNhH2H2Z oFhANZw8rrpTf9jkaSW3BKM32iP+38QF+t6T6F2klBnk5O8g49OQeY8KhgJWlPdmeKRI yddzdxoW82qRc8KfkMuaL4ZVr84OGoEujrHs5MFmZTeViPc820NFG0xpOIHBz5ilLxSN 7PDzz7+DY0TBnz3AezlM7vVW8cQqGxjYZxKS67Nz77UcuAy2MfbSR9jA2W1K+6kVLc4f YTmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=tfc0ew0kAdAAuBXUjDfVzA2fzAypIwJmE3g0LacJX+Y=; b=CGDrrJ/fSYsYzJ69hFyOXDfL2hD4hA0X30V8NrpVIj9U7CnxG/ltVurHzIHY/CZWws 2jgeHzqRN73mFzm3twKKebTm4jkCHg4jXzecNcdtjy4MRXkGwcWk+hdKzQfrvbjrEcpk 2HEzMSgZEbJAwrXtQiIrYZ8SQvtn+BDztoap0WGtZ/OE9P29wI+SUszBK2LWC0X1KzKX cuJY4epjFZBZ66ijcKA5OIzDXoII1b1M6kL+giADCMF/4Ca3G9BNAeKXdi1RtqnwUtN4 t5Bwcbhhq5kSWtA7JriggWskmIY0Wx0v9qTJ9jkxKq8Wp8u7Md/4EIuPkYgZcFGSsgS/ L4Rw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L+2IwLVV; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i127si298565pgc.100.2018.02.26.00.34.01; Mon, 26 Feb 2018 00:34:02 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L+2IwLVV; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752503AbeBZIXS (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:18 -0500 Received: from mail-pg0-f65.google.com ([74.125.83.65]:32782 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751793AbeBZIXN (ORCPT ); Mon, 26 Feb 2018 03:23:13 -0500 Received: by mail-pg0-f65.google.com with SMTP id g12so5911627pgs.0 for ; Mon, 26 Feb 2018 00:23:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=tfc0ew0kAdAAuBXUjDfVzA2fzAypIwJmE3g0LacJX+Y=; b=L+2IwLVV/I7yt8ZRN0CuWj/4LYf+vmJ8g59f02fqiY49+b0GuUfXhNKTDoafP6Azh6 UhP8IGpimoCmEjxYXkayTqWT0xAb4ZkR951W2YUXAsH4IawK+usZVWoj0omgcNFotK+B uiOynSlVLLBNJIeFp+IGzlSMihP6K3s4ROlWE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=tfc0ew0kAdAAuBXUjDfVzA2fzAypIwJmE3g0LacJX+Y=; b=RsUSE1NRVXnD+7n9xl+MMiTjecyQk5CFjBpBqXTCzz4+WvWcPtKpAz+V51BsGUPYS3 8aui0eUj+oy+Dgn7I6UFlqi9RK/4OT47KC4ufVV3ts8o8bi0NVEM7Q+lqa2z6RlBgNQ8 idh+k1qcDpG17bSS0nUFzHHn0w3/kXNUd3YHNVfOb2qoUe1HwvUXGdnqSotFKn5CLGzj 1Dex1jbVWmGwkDGegX1bhN2nuO0PlR+EzXCMw4jbydFjQ9hflHzywa7oVLWCVd3wbVZ5 MOWwxa9TEiS+YFIxZziB29aivsZNQk9TSdW1lQpIZ1NpuzT6h43X3oQOl90cynr1FsVx HBlA== X-Gm-Message-State: APf1xPCZ9hVBEhMlvSNeM8AIMM2i7UeSG2IU81ZfBieIHlrwItXXEG4x 62xgsnB2t+BborCUk28lkYk9ow== X-Received: by 10.99.170.73 with SMTP id x9mr7812380pgo.393.1519633392563; Mon, 26 Feb 2018 00:23:12 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:11 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Mark Rutland , Lorenzo Pieralisi , linux-arm-kernel@lists.infradead.org (open list:POWER STATE COORDINATION INTERFACE (PSCI)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 17/52] drivers/firmware: Expose psci_get_version through psci_ops structure Date: Mon, 26 Feb 2018 16:19:51 +0800 Message-Id: <1519633227-29832-18-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit d68e3ba5303f upstream. Entry into recent versions of ARM Trusted Firmware will invalidate the CPU branch predictor state in order to protect against aliasing attacks. This patch exposes the PSCI "VERSION" function via psci_ops, so that it can be invoked outside of the PSCI driver where necessary. Acked-by: Lorenzo Pieralisi Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- drivers/firmware/psci.c | 2 ++ include/linux/psci.h | 1 + 2 files changed, 3 insertions(+) -- 2.7.4 diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c index 8263429..9a3ce76 100644 --- a/drivers/firmware/psci.c +++ b/drivers/firmware/psci.c @@ -496,6 +496,8 @@ static void __init psci_init_migrate(void) static void __init psci_0_2_set_functions(void) { pr_info("Using standard PSCI v0.2 function IDs\n"); + psci_ops.get_version = psci_get_version; + psci_function_id[PSCI_FN_CPU_SUSPEND] = PSCI_FN_NATIVE(0_2, CPU_SUSPEND); psci_ops.cpu_suspend = psci_cpu_suspend; diff --git a/include/linux/psci.h b/include/linux/psci.h index bdea1cb..6306ab1 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -26,6 +26,7 @@ int psci_cpu_init_idle(unsigned int cpu); int psci_cpu_suspend_enter(unsigned long index); struct psci_operations { + u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); int (*cpu_off)(u32 state); int (*cpu_on)(unsigned long cpuid, unsigned long entry_point); From patchwork Mon Feb 26 08:19:52 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129572 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356834lja; Mon, 26 Feb 2018 00:23:28 -0800 (PST) X-Google-Smtp-Source: AH8x225+QLfbSB/oSrmP2z68w6JNaHp3C6fgssa5vBew5ThpcibkGdNiG/PlwueOGRn4KkH0BTWg X-Received: by 2002:a17:902:968b:: with SMTP id n11-v6mr9837930plp.168.1519633408090; Mon, 26 Feb 2018 00:23:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633408; cv=none; d=google.com; s=arc-20160816; b=tJbcBJlKVLkEnHIwWetmhxrVgn6HEbeouKMlVfR0zf9+Z7hyb434v2C38zMFarY5qO kMbCkdH7T9WfW/yBK1YbnePel0VPAysCGnEJP3rtnTHTSPM9xT1gn6a+F5RXcMLqckhY VD/9TL+iIJx+TeBDHyix7neaHaIJjx3kb3q17C3NztArWyO6ltDQvPTVwjJZK3b51eoi BwLlhJQ06L2UQZW04cQQykpjXFTEJuPakyxlN9nP4xmePdf58Ms9bTO3WOBxU6n691wb f0b92kiN1VLch56UJhI6nLUWL/OA8w4TuoDvZ83Yf2gErw476fdw0l5AHi2W/6XFAVTB yPqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=XfV6Yv3SjvB5RVvJ7haGkLOWLK58ZxDQcMshzGS5NdQ=; b=hmsFSGPn+fffwL0wioAz4/gXVCxiTOHI6LWrmbfagQ7pCBIACutE0ohWXWDBKIyJON 6zMvMF9VFvQoICn6sF6tEdLY3Xcq3SALKoXZLabHggKg+sJ1o29pHI1LeW6Rg+qUD9SC gLCUMH/O+betApqemTyFTqv2K+ZWpAzbNK4PsVB9hUy3+a68dI++fbpTIFdXe+c8LsLD uAWXNvKt/keks2TziFj5nSoLwFYJ28jFEEMXWG+jdLImBnOO1xz8H9fcAaYA8Ah60xIr 5Lx8lOyuNUsN6zq+KUxOvoFet/xsUKtoNjuPyQcWk0oG5+a9y/di29MHcmvfhp1yk3+6 zbVg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GUnDuslp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m12-v6si3738150pll.351.2018.02.26.00.23.27; Mon, 26 Feb 2018 00:23:28 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GUnDuslp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752531AbeBZIXZ (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:25 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:38561 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751793AbeBZIXT (ORCPT ); Mon, 26 Feb 2018 03:23:19 -0500 Received: by mail-pg0-f67.google.com with SMTP id l24so5915022pgc.5 for ; Mon, 26 Feb 2018 00:23:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=XfV6Yv3SjvB5RVvJ7haGkLOWLK58ZxDQcMshzGS5NdQ=; b=GUnDuslpSiChZ+QIgEJzPMtjXw0pn/sfSRH6ZkzKshR10JThMZJJjWh/Zb7/BWrS+U /I68OuzXRLS5btS73LM3MYhwx0Mxltux2/+dOlXJQborMenSXZwjwBfN2CrttmqMVs78 p8WRyf6jWZok/ewRYnJESmK4NpvAdfpUpDgSw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=XfV6Yv3SjvB5RVvJ7haGkLOWLK58ZxDQcMshzGS5NdQ=; b=gNgHVCIp6WtUazM45SHsvtIttT7kGqk+zcf+jprshJFdTvu5IFzDnz5Vw3XtsbKNRb l+oaJAnc6GZLqGlRRZtXtyvmX70iLa4JcCbgEgLof8EHeNgQeSA4vPGzKjPR4Fv8ClOX 63pt3P8shhhjpSTbeKFoXlyFrSWcS1Bz28zJkGUumrm6TTOSyxoB/s7GTLtOPjkuLT4D UAaclTArmeTNZKHhc2tcTagTBDv8Az6q0n0L4zydub5a02YM4NvOyr1qtpaTc4A8r/Cl hN66MAaMAPWFh+7B5wbWSdVztjKTNQapqfcjevzXzoEKrwYsywpcqHEA/XDXK2Z86cle agzQ== X-Gm-Message-State: APf1xPCOqLBlng9E5+mX2EGisL+vlN16HMiF4opxvMmfIgMb6/iYwgay 83QTK0FtGPfTciNTeO7hDhgplA== X-Received: by 10.98.91.5 with SMTP id p5mr9794882pfb.163.1519633399033; Mon, 26 Feb 2018 00:23:19 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:18 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Cc: James Morse Subject: [PATCH 18/52] arm64: cpufeature: __this_cpu_has_cap() shouldn't stop early Date: Mon, 26 Feb 2018 16:19:52 +0800 Message-Id: <1519633227-29832-19-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: James Morse commit edf298cfce47 upstream. Alex Shi rewrite this commit on func this_cpu_has_cap(). The following commit log is still meaningful. this_cpu_has_cap() tests caps->desc not caps->matches, so it stops walking the list when it finds a 'silent' feature, instead of walking to the end of the list. Prior to v4.6's 644c2ae198412 ("arm64: cpufeature: Test 'matches' pointer to find the end of the list") we always tested desc to find the end of a capability list. This was changed for dubious things like PAN_NOT_UAO. v4.7's e3661b128e53e ("arm64: Allow a capability to be checked on single CPU") added this_cpu_has_cap() using the old desc style test. CC: Suzuki K Poulose Reviewed-by: Suzuki K Poulose Acked-by: Marc Zyngier Signed-off-by: James Morse Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/include/asm/futex.h | 1 + arch/arm64/kernel/cpufeature.c | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/include/asm/futex.h b/arch/arm64/include/asm/futex.h index 1943cf6..718140a 100644 --- a/arch/arm64/include/asm/futex.h +++ b/arch/arm64/include/asm/futex.h @@ -20,6 +20,7 @@ #include #include +#include #include diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 3a129d4..fcb67e0 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1063,8 +1063,8 @@ bool this_cpu_has_cap(unsigned int cap) if (WARN_ON(preemptible())) return false; - for (caps = arm64_features; caps->desc; caps++) - if (caps->capability == cap && caps->matches) + for (caps = arm64_features; caps->matches; caps++) + if (caps->capability == cap) return caps->matches(caps, SCOPE_LOCAL_CPU); return false; From patchwork Mon Feb 26 08:19:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129573 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3356917lja; Mon, 26 Feb 2018 00:23:35 -0800 (PST) X-Google-Smtp-Source: AH8x227KQDVd3Vb/Kvy+hDY7A9cbhLU2+hRdgueRLyToBb+2WV5FQAcadjlSVXMOJ21wARt5hxUM X-Received: by 10.98.163.67 with SMTP id s64mr9847800pfe.67.1519633415117; Mon, 26 Feb 2018 00:23:35 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633415; cv=none; d=google.com; s=arc-20160816; b=lciz7kOby5v1zw+osgpLAJEBbKa0XXwhclhINnPRO+Js9cyQftoXVxnOvDXkpUhGIl 6ZKHJW8tYZ88F70zU0zUDiiPXEmAi+/CQMxd0vLNyCf8jZp9obAN4btAXZl2szOj3UBe im+NH9jlofFyU27067c74Hj/OMnWBNBSOeuYt6LbWB4W+CnTXrH8MMTiu0evtRy8jEZQ AA1ui6OXZunbo05YktPoyKh50XNjnDuCyvH6jto+AM2duy289Z0/rEmLCwSABXf+lCep bPblw7eMxwiNYk7k2a7Yw/avQWpd95s6YCSA/HCRbtRD4tOmfPipPMOEzW/wW65uaDkK 2gcg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=lZSZ7mN/jhpd6MgLblCtRDrj6c7/jUPbXNYPUP0WgH0=; b=AJYBqzGgKEgPHeL1O1BFVNrSvO6vKnWOxrxle7Zrelf9xKV8lM9wiRi/kE6FYyfBUb BPhAs5j4O6OtsDFGnh8wi6s4eSe8TY5F65RXKAnyHqFBigFx9I0MwlfY5RtNA8Y62Ht5 5WCsDooqHljBT9/5NrMLDTr0W9feTiYqthPIuS0ADOhkwxY7hbnTS9sUva81MX5FIo9r xYs6zwtnXgIFc8XTNwCNBoT81dfCJ3stKXEZa1mIuonlO10SWHIkVxJWHa/TD/rK2QNq 1sozVbyPLRM8AguG1Q8JT6usFuTxIa0u0LpBLmP3zHWez5E8UUpPoFXOxi8Um0FPPHYV wGTQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C0FlJHD/; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m12-v6si3738150pll.351.2018.02.26.00.23.34; Mon, 26 Feb 2018 00:23:35 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C0FlJHD/; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752536AbeBZIXb (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:31 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:33282 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752317AbeBZIX0 (ORCPT ); Mon, 26 Feb 2018 03:23:26 -0500 Received: by mail-pf0-f194.google.com with SMTP id q13so6223074pff.0 for ; Mon, 26 Feb 2018 00:23:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=lZSZ7mN/jhpd6MgLblCtRDrj6c7/jUPbXNYPUP0WgH0=; b=C0FlJHD/X8UDYz8E8ZjLvGAnsiBI0Q2HPR3JReFTRfeB7pziFNxym5iMJwS9I6YgcT 3WGC1QFl29hNlpFY678mRq/Sv+tJuaHM13JZLhinQduhf+yW3HzhiyRaCofA7P5km2Z5 dYCKmOqr/w9OPen3fvnEVWsELg0hgewaLIX7I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=lZSZ7mN/jhpd6MgLblCtRDrj6c7/jUPbXNYPUP0WgH0=; b=WR9+ABLlfoKW/JP7HIOttsQk+jMKvxGyocgt2KY9ziZsJ27K/NKu6Zrin5/kSRBssZ DGliZSQeqDDmsApy0jZMIjhFW90kwyNXNCQdNJpmpVY6inEKkXDtRV3o6aOmVjAdaqv0 EMHxqVHE2qBX9ePkP/jcIf/J+gcu7+r4yO98qfZkhGlV+QNucrltkKL5Uxs6I+e1XGKa NeQkz9OZnJdcZYH6Hu/C4ZygS9ywuor5VqbEpTjuo7k4E7FuBsWPr7W3pCn9SQR1NKyH uuJsiICC7m7LBVCzHdnBXA9RaJ71Rp61JnuE30QiI0F71rAXi/X08ernTwGZy7+Lr3G3 HbQA== X-Gm-Message-State: APf1xPBritQKo+hnO75TUqrgZ0ci6Gp3byERX39bKgakvbGjTg1QVuG/ 1roTrffJOanLZKm6+IwzVYtMDA== X-Received: by 10.167.129.67 with SMTP id d3mr9821874pfn.108.1519633405731; Mon, 26 Feb 2018 00:23:25 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.19 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:25 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 19/52] arm64: cpu_errata: Allow an erratum to be match for all revisions of a core Date: Mon, 26 Feb 2018 16:19:53 +0800 Message-Id: <1519633227-29832-20-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 06f1494f837 upstream. Some minor erratum may not be fixed in further revisions of a core, leading to a situation where the workaround needs to be updated each time an updated core is released. Introduce a MIDR_ALL_VERSIONS match helper that will work for all versions of that MIDR, once and for all. Acked-by: Thomas Gleixner Acked-by: Mark Rutland Acked-by: Daniel Lezcano Reviewed-by: Suzuki K Poulose Signed-off-by: Marc Zyngier Signed-off-by: Alex Shi --- arch/arm64/kernel/cpu_errata.c | 7 +++++++ 1 file changed, 7 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index b75e917..c66a673c 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -53,6 +53,13 @@ static int cpu_enable_trap_ctr_access(void *__unused) .midr_range_min = min, \ .midr_range_max = max +#define MIDR_ALL_VERSIONS(model) \ + .def_scope = SCOPE_LOCAL_CPU, \ + .matches = is_affected_midr_range, \ + .midr_model = model, \ + .midr_range_min = 0, \ + .midr_range_max = (MIDR_VARIANT_MASK | MIDR_REVISION_MASK) + const struct arm64_cpu_capabilities arm64_errata[] = { #if defined(CONFIG_ARM64_ERRATUM_826319) || \ defined(CONFIG_ARM64_ERRATUM_827319) || \ From patchwork Mon Feb 26 08:19:56 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129575 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357142lja; Mon, 26 Feb 2018 00:23:54 -0800 (PST) X-Google-Smtp-Source: AH8x225Loe0Qz+DFbFyETwWz2QNymYpnDw93Xx2v71FyUmI+AHuqjje0gncHnJiDBlK5r4N/TInv X-Received: by 2002:a17:902:6e0f:: with SMTP id u15-v6mr9660935plk.78.1519633434487; Mon, 26 Feb 2018 00:23:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633434; cv=none; d=google.com; s=arc-20160816; b=NtfIecZdztr54mZhI0hmmdkUjyjvEr5ywAMfDwEo+2mcjFZH7yBBx8Jat70+I70OA2 zFolu6m5s1gdBFqwFWEHwZmyyOHwNoa5hAZ/jbqZhXqPaHAoH3ku1D0zXP8lwTRJ8E6U lnW4QWED0AGfXp3J+M37uGboqNsupuZlcw/287gmz1OfBZp3BvNWeox6xE4VmMHgKZPp byFKTUQ7W3DPTkHX1HQR4FPdGMP/ZGkhN6/MePHn6pFdFUBgDCAJBhjXuBb7giXco3uU 8ETuxzgNhwfaBRAEEcqnUnx3+jPqaVXBIc+UarN4HN41KokRrAx8y0qY6VXpOT+GOpLK b+pA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=vlTjjzmQguibQ42hHx0fE9h4nxyZ4I8LAAjWye+MNJs=; b=EcJxTdsMHRhGoVjx3vDist1rPqdnbwr71pIM43UkSJtGGJbo9hjWGQfL2qoshvtGwY QNgn5ooaZtf/rkDdkDjOFTf+d3+iimu7f6UDtUIAhjDYv8kcWlef7gFoK3CJYyVC6g8N oulEG4XLzGmwGf6ZiRqV8yoPUNEt5rj5QxWCP/JeGJ3F/ravx+QeZ1BmPrMleo2PiCMp XHI+khbC5VrI6j0yDpQysXjuPlFs2DLKNUBpEgA9CxMyrTXc/pDrfb10i9XpsN1VYvJi eCD5dhV1Ezw0xrDHiLUs2/kha4xMFTG6nJMedOmSuXCvzvxCpBHHSy3KXoKEnI18sXVP 6s+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H6XzJ+Lr; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x7si6415744pfb.298.2018.02.26.00.23.54; Mon, 26 Feb 2018 00:23:54 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H6XzJ+Lr; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751908AbeBZIXv (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:51 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:41191 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752009AbeBZIXq (ORCPT ); Mon, 26 Feb 2018 03:23:46 -0500 Received: by mail-pf0-f194.google.com with SMTP id f80so1120168pfa.8 for ; Mon, 26 Feb 2018 00:23:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=vlTjjzmQguibQ42hHx0fE9h4nxyZ4I8LAAjWye+MNJs=; b=H6XzJ+LrfMw30lEmQuML6DlUcJu1aA7EtJTzC3GoVnxPmrxa9iUpyY+8DS1Z0bEIog dPU0d56WZC2pgalAdZ8ZhGfFyJAh3jm2cf5Er8/RzDGWYWvKXL27UlXW7SBUU466fk1m cePvDDJoKWzZ99ITa1PJqQdKFTrqF5wGxNffQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=vlTjjzmQguibQ42hHx0fE9h4nxyZ4I8LAAjWye+MNJs=; b=uc2VrHE7ma90/F/0RtUGwbJ1YP4Oqip943IyfQvXHn4DpT21lXAE4kQBKwCinkSBSO SUtIUaM3AIsm2xOE0Vj8An7YjMXPJl+hb2JOeJuF8gexz8Vg8/LkE8UtvmdWw7/eOpaU NMxIfX2v1ucbgsYrU+ko2/EIUaMMqDF1re59Lmu3GlUREpQisKRbdaSiuN3HeElR7iM+ ATcgtdguSVOimQhZcnqgpew9y/h3MkxCSikpQmxWm+GK2gxRlDWXR7sepFX6gMLeKpD+ /QNfrVkLVypSut7Mqg/5CEdJuL5fWNQN9jgLpX+6xy7PhFd2esZprADK082IlFuPyftT GVxA== X-Gm-Message-State: APf1xPBrEKbQo7Cv80k/UcjQzhBxqoWsbjM/TSM3rQTIZR5V8uZfJZRb nYYsGDoqQBLAejhfsEv+NbUaMg== X-Received: by 10.98.96.70 with SMTP id u67mr9936386pfb.66.1519633425734; Mon, 26 Feb 2018 00:23:45 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.40 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:45 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 22/52] arm64: Move post_ttbr_update_workaround to C code Date: Mon, 26 Feb 2018 16:19:56 +0800 Message-Id: <1519633227-29832-23-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 95e3de3590e3 upstream. We will soon need to invoke a CPU-specific function pointer after changing page tables, so move post_ttbr_update_workaround out into C code to make this possible. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: don't include PAN related changes arch/arm64/include/asm/assembler.h arch/arm64/kernel/entry.S arch/arm64/mm/proc.S --- arch/arm64/mm/context.c | 9 +++++++++ arch/arm64/mm/proc.S | 3 +-- 2 files changed, 10 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index efcf1f7..32eeabe91 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -224,6 +224,15 @@ void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) cpu_switch_mm(mm->pgd, mm); } +/* Errata workaround post TTBRx_EL1 update. */ +asmlinkage void post_ttbr_update_workaround(void) +{ + asm(ALTERNATIVE("nop; nop; nop", + "ic iallu; dsb nsh; isb", + ARM64_WORKAROUND_CAVIUM_27456, + CONFIG_CAVIUM_ERRATUM_27456)); +} + static int asids_init(void) { asid_bits = get_cpu_asid_bits(); diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index c2adb0c..cca061a 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -136,8 +136,7 @@ ENTRY(cpu_do_switch_mm) bfi x0, x1, #48, #16 // set the ASID msr ttbr0_el1, x0 // set TTBR0 isb - post_ttbr0_update_workaround - ret + b post_ttbr_update_workaround // Back to C code... ENDPROC(cpu_do_switch_mm) .pushsection ".idmap.text", "ax" From patchwork Mon Feb 26 08:19:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129576 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357218lja; Mon, 26 Feb 2018 00:24:01 -0800 (PST) X-Google-Smtp-Source: AH8x2251tYiGWyJbNDE+mG9/90WzQSrSktEsxDpoC1/gprEnYtP7v12l0Qdz5p5FvlescTtH+hmM X-Received: by 10.101.92.72 with SMTP id v8mr7745968pgr.153.1519633441301; Mon, 26 Feb 2018 00:24:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633441; cv=none; d=google.com; s=arc-20160816; b=K2qB2u+BSB/u6qVKSWD6FZuxfAn7o5CHgyNT9ao5ymMaiKRI7hSQSeTQZkVcLZgmTL GcpHeTBgekC360n/PYECH8dZwqhH8czlgbAk5BChP4LnjxzgCmeTA4Sgn+SeNl+FW345 208xA09VFoIhdN/EgIcx0xk4DKJ5kmfMxZvCRK8hQDn4HEwPBlIPUWgLCJ+gmX78IGhc fmIO7/lpLfXE4C9fKR6301MEmzz8lSqHhAEipUXwb8vuzFXw2SFSl9CJNiqL+XJII8DN U+jZzXHBbB2rVFLklx1zotrkHC2lwhN2fGt/u1/O8p4dY3vplGU5iOoGrdMq1+mhkllA d54g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=jfEkyNuRFI0AGI2zyS0Weuuew1S0Dlud8Iqkt87+Caw=; b=UP7bDCaruiWTEuEyHxbbSjnRoolERaiXofC6yZ8KXW4d0C/U/hxXOR0xh69HEIzVhE buole8VwqnQaSi+hL8NFobwVUuuTS0CoiYsiuvXkRlpIGyNn2DfndROGEi/X5tLLdx59 0LdRVKbV0VdbLBQ1ODjh+vDflVqzxxleTajAFrEDeFTAA7BiHZy5Bfr/svM2KAORBTQZ vsfP9ncvECrEgTKa19hLk0b2LiviDv3BWI17IOCS5G7oGr+783UULUIx0Q6cKCUyq5nT KbKdicKEH6zpaTJEDBzPydowDCAjjzF2ajOL0/3qFWD/15v0zY4W37I6R4E29broxusm eVBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C9hGrqva; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x7si6415744pfb.298.2018.02.26.00.24.01; Mon, 26 Feb 2018 00:24:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=C9hGrqva; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752569AbeBZIX6 (ORCPT + 28 others); Mon, 26 Feb 2018 03:23:58 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:42709 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752558AbeBZIXx (ORCPT ); Mon, 26 Feb 2018 03:23:53 -0500 Received: by mail-pf0-f193.google.com with SMTP id a16so395956pfn.9 for ; Mon, 26 Feb 2018 00:23:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=jfEkyNuRFI0AGI2zyS0Weuuew1S0Dlud8Iqkt87+Caw=; b=C9hGrqvaKWSEOIF8A2ucHqsWd54O7Iq7sh0ck0PB0U4zvCdvMp5DDIkN7Bv6/CP4vV Nftf1U7/zRzvp2cHW7yPC+rVaIPh6tM500I1EyRvDnRnvAdtIOUE8z3jD0W+WmV5mdIG Ax/PNwlYD2tWUmuaEEkOt//BL+sNJEGV6fc7A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=jfEkyNuRFI0AGI2zyS0Weuuew1S0Dlud8Iqkt87+Caw=; b=fE7KIZ14VVGXTt1gf2WdT33Mf3az0NVV1svXyIMDN//jdUUj7rQmtzZJPjM+sdvK5u rfgIsDgF1UxHjPZG9C1RuOXimuRpLnqDzn1TKfijn7ZlibUvXtfyTysPo6Xub2fVA28b NfpVLAsF3vi7MJRqHVR6FE/e7X2epp9EXkitFcsSP1klRcf036MwSg6mabm8cs3OkmUO aHuUIxdgIKULRQUQgEvOaVQyGAUEhiCEhSAf1KZ9ldEY58bxe24qf6d1cpEhik3D0WGV Hb2YLpan00cjs07hi1fmb82m65IXpB+wdpkVH8cddeQ2WmuQnddHfUXfQe4/N7I8Xrk5 /b+g== X-Gm-Message-State: APf1xPAHSQBU33vek51dBlOuqRp9JymXKRJ2fYspTTVJ/a04Ylmq2twS 8aujY4bChA/qmXbrYB0gnrmyHGLbPLo= X-Received: by 10.167.129.67 with SMTP id d3mr9822880pfn.108.1519633433125; Mon, 26 Feb 2018 00:23:53 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:52 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 23/52] arm64: Add skeleton to harden the branch predictor against aliasing attacks Date: Mon, 26 Feb 2018 16:19:57 +0800 Message-Id: <1519633227-29832-24-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit 0f15adbb2861 upstream. Aliasing attacks against CPU branch predictors can allow an attacker to redirect speculative control flow on some CPUs and potentially divulge information from one context to another. This patch adds initial skeleton code behind a new Kconfig option to enable implementation-specific mitigations against these attacks for CPUs that are affected. Co-developed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: expand enable_da_f in entry.S use 5 parameters ARM64_FTR_BITS() add percpu.h in mm_types.h for percpu functions use cpus_have_cap instead of cpus_have_const_cap arch/arm64/Kconfig arch/arm64/include/asm/cpucaps.h arch/arm64/include/asm/mmu.h arch/arm64/include/asm/sysreg.h arch/arm64/kernel/cpufeature.c arch/arm64/kernel/entry.S arch/arm64/mm/fault.c --- arch/arm64/Kconfig | 17 +++++++++ arch/arm64/include/asm/cpucaps.h | 3 +- arch/arm64/include/asm/mmu.h | 38 +++++++++++++++++++++ arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kernel/Makefile | 4 +++ arch/arm64/kernel/bpi.S | 55 +++++++++++++++++++++++++++++ arch/arm64/kernel/cpu_errata.c | 74 ++++++++++++++++++++++++++++++++++++++++ arch/arm64/kernel/cpufeature.c | 1 + arch/arm64/kernel/entry.S | 8 +++-- arch/arm64/mm/context.c | 2 ++ arch/arm64/mm/fault.c | 17 +++++++++ include/linux/mm_types.h | 1 + 12 files changed, 217 insertions(+), 4 deletions(-) create mode 100644 arch/arm64/kernel/bpi.S -- 2.7.4 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7769c2e..0c4be63 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -733,6 +733,23 @@ config FORCE_MAX_ZONEORDER However for 4K, we choose a higher default value, 11 as opposed to 10, giving us 4M allocations matching the default size used by generic code. +config HARDEN_BRANCH_PREDICTOR + bool "Harden the branch predictor against aliasing attacks" if EXPERT + default y + help + Speculation attacks against some high-performance processors rely on + being able to manipulate the branch predictor for a victim context by + executing aliasing branches in the attacker context. Such attacks + can be partially mitigated against by clearing internal branch + predictor state and limiting the prediction logic in some situations. + + This config option will take CPU-specific actions to harden the + branch predictor against aliasing attacks and may rely on specific + instruction sequences or control bits being set by the system + firmware. + + If unsure, say Y. + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index 87b4465..f8b7799 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -34,7 +34,8 @@ #define ARM64_HAS_32BIT_EL0 13 #define ARM64_HYP_OFFSET_LOW 14 #define ARM64_MISMATCHED_CACHE_LINE_SIZE 15 +#define ARM64_HARDEN_BRANCH_PREDICTOR 16 -#define ARM64_NCAPS 16 +#define ARM64_NCAPS 17 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu.h b/arch/arm64/include/asm/mmu.h index b075140..203974c 100644 --- a/arch/arm64/include/asm/mmu.h +++ b/arch/arm64/include/asm/mmu.h @@ -28,6 +28,44 @@ typedef struct { */ #define ASID(mm) ((mm)->context.id.counter & 0xffff) + +typedef void (*bp_hardening_cb_t)(void); + +struct bp_hardening_data { + int hyp_vectors_slot; + bp_hardening_cb_t fn; +}; + +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +extern char __bp_harden_hyp_vecs_start[], __bp_harden_hyp_vecs_end[]; + +DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); + +static inline struct bp_hardening_data *arm64_get_bp_hardening_data(void) +{ + return this_cpu_ptr(&bp_hardening_data); +} + +static inline void arm64_apply_bp_hardening(void) +{ + struct bp_hardening_data *d; + + if (!cpus_have_cap(ARM64_HARDEN_BRANCH_PREDICTOR)) + return; + + d = arm64_get_bp_hardening_data(); + if (d->fn) + d->fn(); +} +#else +static inline struct bp_hardening_data *arm64_get_bp_hardening_data(void) +{ + return NULL; +} + +static inline void arm64_apply_bp_hardening(void) { } +#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ + extern void paging_init(void); extern void bootmem_init(void); extern void __iomem *early_io_map(phys_addr_t phys, unsigned long virt); diff --git a/arch/arm64/include/asm/sysreg.h b/arch/arm64/include/asm/sysreg.h index 7393cc7..e91710f 100644 --- a/arch/arm64/include/asm/sysreg.h +++ b/arch/arm64/include/asm/sysreg.h @@ -117,6 +117,7 @@ #define ID_AA64ISAR0_AES_SHIFT 4 /* id_aa64pfr0 */ +#define ID_AA64PFR0_CSV2_SHIFT 56 #define ID_AA64PFR0_GIC_SHIFT 24 #define ID_AA64PFR0_ASIMD_SHIFT 20 #define ID_AA64PFR0_FP_SHIFT 16 diff --git a/arch/arm64/kernel/Makefile b/arch/arm64/kernel/Makefile index 7d66bba..74b8fd8 100644 --- a/arch/arm64/kernel/Makefile +++ b/arch/arm64/kernel/Makefile @@ -51,6 +51,10 @@ arm64-obj-$(CONFIG_HIBERNATION) += hibernate.o hibernate-asm.o arm64-obj-$(CONFIG_KEXEC) += machine_kexec.o relocate_kernel.o \ cpu-reset.o +ifeq ($(CONFIG_KVM),y) +arm64-obj-$(CONFIG_HARDEN_BRANCH_PREDICTOR) += bpi.o +endif + obj-y += $(arm64-obj-y) vdso/ probes/ obj-m += $(arm64-obj-m) head-y := head.o diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S new file mode 100644 index 0000000..06a931e --- /dev/null +++ b/arch/arm64/kernel/bpi.S @@ -0,0 +1,55 @@ +/* + * Contains CPU specific branch predictor invalidation sequences + * + * Copyright (C) 2018 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +.macro ventry target + .rept 31 + nop + .endr + b \target +.endm + +.macro vectors target + ventry \target + 0x000 + ventry \target + 0x080 + ventry \target + 0x100 + ventry \target + 0x180 + + ventry \target + 0x200 + ventry \target + 0x280 + ventry \target + 0x300 + ventry \target + 0x380 + + ventry \target + 0x400 + ventry \target + 0x480 + ventry \target + 0x500 + ventry \target + 0x580 + + ventry \target + 0x600 + ventry \target + 0x680 + ventry \target + 0x700 + ventry \target + 0x780 +.endm + + .align 11 +ENTRY(__bp_harden_hyp_vecs_start) + .rept 4 + vectors __kvm_hyp_vector + .endr +ENTRY(__bp_harden_hyp_vecs_end) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 8de43799..0e07893 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -46,6 +46,80 @@ static int cpu_enable_trap_ctr_access(void *__unused) return 0; } +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR +#include +#include + +DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); + +#ifdef CONFIG_KVM +static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + void *dst = lm_alias(__bp_harden_hyp_vecs_start + slot * SZ_2K); + int i; + + for (i = 0; i < SZ_2K; i += 0x80) + memcpy(dst + i, hyp_vecs_start, hyp_vecs_end - hyp_vecs_start); + + flush_icache_range((uintptr_t)dst, (uintptr_t)dst + SZ_2K); +} + +static void __install_bp_hardening_cb(bp_hardening_cb_t fn, + const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + static int last_slot = -1; + static DEFINE_SPINLOCK(bp_lock); + int cpu, slot = -1; + + spin_lock(&bp_lock); + for_each_possible_cpu(cpu) { + if (per_cpu(bp_hardening_data.fn, cpu) == fn) { + slot = per_cpu(bp_hardening_data.hyp_vectors_slot, cpu); + break; + } + } + + if (slot == -1) { + last_slot++; + BUG_ON(((__bp_harden_hyp_vecs_end - __bp_harden_hyp_vecs_start) + / SZ_2K) <= last_slot); + slot = last_slot; + __copy_hyp_vect_bpi(slot, hyp_vecs_start, hyp_vecs_end); + } + + __this_cpu_write(bp_hardening_data.hyp_vectors_slot, slot); + __this_cpu_write(bp_hardening_data.fn, fn); + spin_unlock(&bp_lock); +} +#else +static void __install_bp_hardening_cb(bp_hardening_cb_t fn, + const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + __this_cpu_write(bp_hardening_data.fn, fn); +} +#endif /* CONFIG_KVM */ + +static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry, + bp_hardening_cb_t fn, + const char *hyp_vecs_start, + const char *hyp_vecs_end) +{ + u64 pfr0; + + if (!entry->matches(entry, SCOPE_LOCAL_CPU)) + return; + + pfr0 = read_cpuid(ID_AA64PFR0_EL1); + if (cpuid_feature_extract_unsigned_field(pfr0, ID_AA64PFR0_CSV2_SHIFT)) + return; + + __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end); +} +#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ + #define MIDR_RANGE(model, min, max) \ .def_scope = SCOPE_LOCAL_CPU, \ .matches = is_affected_midr_range, \ diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 5c41ef6..6e7fda3 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -98,6 +98,7 @@ static const struct arm64_ftr_bits ftr_id_aa64pfr0[] = { ARM64_FTR_BITS(FTR_STRICT, FTR_EXACT, ID_AA64PFR0_GIC_SHIFT, 4, 0), S_ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_ASIMD_SHIFT, 4, ID_AA64PFR0_ASIMD_NI), S_ARM64_FTR_BITS(FTR_STRICT, FTR_LOWER_SAFE, ID_AA64PFR0_FP_SHIFT, 4, ID_AA64PFR0_FP_NI), + ARM64_FTR_BITS(FTR_NONSTRICT, FTR_LOWER_SAFE, ID_AA64PFR0_CSV2_SHIFT, 4, 0), /* Linux doesn't care about the EL3 */ ARM64_FTR_BITS(FTR_NONSTRICT, FTR_EXACT, ID_AA64PFR0_EL3_SHIFT, 4, 0), ARM64_FTR_BITS(FTR_STRICT, FTR_EXACT, ID_AA64PFR0_EL2_SHIFT, 4, 0), diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 0a27e12..bdb0139 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -549,13 +549,15 @@ el0_ia: * Instruction abort handling */ mrs x26, far_el1 - // enable interrupts before calling the main handler - enable_dbg_and_irq + msr daifclr, #(8 | 4 | 1) +#ifdef CONFIG_TRACE_IRQFLAGS + bl trace_hardirqs_off +#endif ct_user_exit mov x0, x26 mov x1, x25 mov x2, sp - bl do_mem_abort + bl do_el0_ia_bp_hardening b ret_to_user el0_fpsimd_acc: /* diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 32eeabe91..afc9266 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -231,6 +231,8 @@ asmlinkage void post_ttbr_update_workaround(void) "ic iallu; dsb nsh; isb", ARM64_WORKAROUND_CAVIUM_27456, CONFIG_CAVIUM_ERRATUM_27456)); + + arm64_apply_bp_hardening(); } static int asids_init(void) diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 4df70c9..c95b194 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -590,6 +590,23 @@ asmlinkage void __exception do_mem_abort(unsigned long addr, unsigned int esr, arm64_notify_die("", regs, &info, esr); } +asmlinkage void __exception do_el0_ia_bp_hardening(unsigned long addr, + unsigned int esr, + struct pt_regs *regs) +{ + /* + * We've taken an instruction abort from userspace and not yet + * re-enabled IRQs. If the address is a kernel address, apply + * BP hardening prior to enabling IRQs and pre-emption. + */ + if (addr > TASK_SIZE) + arm64_apply_bp_hardening(); + + local_irq_enable(); + do_mem_abort(addr, esr, regs); +} + + /* * Handle stack alignment exceptions. */ diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index e8471c2..15a82f3 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -13,6 +13,7 @@ #include #include #include +#include #include #include From patchwork Mon Feb 26 08:19:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129577 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357277lja; Mon, 26 Feb 2018 00:24:07 -0800 (PST) X-Google-Smtp-Source: AH8x224LDFrqv0xZmyTp8+TbGrW6FjHB/cp7bkGnUqSPaxI0Cyr4RoyDpS4SgioYaWzoUu7YTuQy X-Received: by 2002:a17:902:b7c2:: with SMTP id v2-v6mr9738511plz.351.1519633447465; Mon, 26 Feb 2018 00:24:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633447; cv=none; d=google.com; s=arc-20160816; b=vP6hL/KMIJVfqE0w6FsM6df/OdZKE07Oyzert0whdZ1cinmr9GyFS5W2UDvU5ThmKO MWb6v9Xgn3qm7CG6BpjVOxbrzvRheyJ/ToPvTM1v8fQ8SAKBYz9UebqPjXn9U19PkGCl gFYrzft5KHeXn4F/2o4fpicGK5ZvLvtk/5rF84kxhx+7JMTn0Nefq0bCP2xTbgIpyzw6 k+kn+GbQx6/DecX4eCLHgubflu9et60tZjBoTp1unx2axsEGKF2yCHY+kSk/UQa/Q/Bk nDPQcYgNknlEM9YzQoX1WNXLb6NOcQxntmMjK1Z1Z06RM0eRHWTRYvWiyseBIt9YBuyq 5gCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=i1j7Y/8yxFvtha7QKxycJLsEJDbrUtAASAguHbCEvfE=; b=om8gLYkY3+HI3kH62izikZwHQ3Fjk+oZm1t2j+GRB77Nv5GYXmyCZrx4iJvSrRrrij EJSDG4+oTCsxkkCKzf5aoyiQtVXxglDV1eM9IkFZOTmJuExI0FoYt99E4n7XfbMxH3zC texWZlGbm6OkVAjMHWzpNEe2ZuUoAseUanVdQ5zLRnVkFIn8Gff00D+ad9e0fquMC6qw fIvnMuu5MftwwRGZQrIxnmjF3q2srNWOUdwfXyjZBFVIQY98+0KaVh7glFres54++oLD UDmZCB38qsRBc5c6uUCzTxYKHbtqr1cqPupe25bAEXfr1p4kfwRb91iZlCOnl2kEmbfd QFWg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CEJiCbWl; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b11si5249076pgr.612.2018.02.26.00.24.07; Mon, 26 Feb 2018 00:24:07 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CEJiCbWl; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752579AbeBZIYF (ORCPT + 28 others); Mon, 26 Feb 2018 03:24:05 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:40637 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752558AbeBZIYA (ORCPT ); Mon, 26 Feb 2018 03:24:00 -0500 Received: by mail-pl0-f66.google.com with SMTP id i6so8867570plt.7 for ; Mon, 26 Feb 2018 00:24:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=i1j7Y/8yxFvtha7QKxycJLsEJDbrUtAASAguHbCEvfE=; b=CEJiCbWleySP5jLeh92MpVfuwucYcYtIqYEyq+tq/WZTWIrLSmxxBwsx74HHR2IRQI As4BwTeF+xm+NE7vklxC9XWkSvjwRZqmL/IafolLCeHMEZ2ivDMDiagMG/mMaD2piZ2f N571MwRlH3Qj5k9k2SxMmuDIxo1FYSlbR8830= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=i1j7Y/8yxFvtha7QKxycJLsEJDbrUtAASAguHbCEvfE=; b=ETgxPqvAiWlig5h6VZgMRN3ounmUoNFzkdgcRM+5lL7+4y1Rc3ghtVTvKrh354w2eo KDhZAv0f4ogKD3SK4ywK0DBF5piyv7lbOMFvo1jTybSrJg1fFsNzfNHRk6rPuNkmcLNo NBVbIO6jM+/TdCnfgDWPy9zu8k4wVl4WLF2pSk9zcrfupXT7/lC1BKl8ZDZiLDZQi+Mn dr/7BYtnhpKBDlS8vKatBVPD/jl8FSlHG1l1E5F+M0IDHv+XTcbTzrBgD6Nbay6akXQ/ FYXTaCuXbnbEfhJStkdYLWXZARvvqlDfzjlcdHU7TDUDxxu93sZw1gmMI+E8sbfEq04t VB9g== X-Gm-Message-State: APf1xPDx39NyUO/DwIdjOAYV1u6em/FPEKXvgarJrOmMmdnFETW9iwAk 3uRACesLbCprcFWzl1h6MzWuKw== X-Received: by 2002:a17:902:3341:: with SMTP id a59-v6mr4855144plc.20.1519633439665; Mon, 26 Feb 2018 00:23:59 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.23.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:23:59 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 24/52] arm64: Move BP hardening to check_and_switch_context Date: Mon, 26 Feb 2018 16:19:58 +0800 Message-Id: <1519633227-29832-25-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit a8e4c0a919ae upstream. We call arm64_apply_bp_hardening() from post_ttbr_update_workaround, which has the unexpected consequence of being triggered on every exception return to userspace when ARM64_SW_TTBR0_PAN is selected, even if no context switch actually occured. This is a bit suboptimal, and it would be more logical to only invalidate the branch predictor when we actually switch to a different mm. In order to solve this, move the call to arm64_apply_bp_hardening() into check_and_switch_context(), where we're guaranteed to pick a different mm context. Acked-by: Will Deacon Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no sw pan in arch/arm64/mm/context.c --- arch/arm64/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index afc9266..36416e4 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -221,6 +221,7 @@ void check_and_switch_context(struct mm_struct *mm, unsigned int cpu) raw_spin_unlock_irqrestore(&cpu_asid_lock, flags); switch_mm_fastpath: + arm64_apply_bp_hardening(); cpu_switch_mm(mm->pgd, mm); } @@ -231,8 +232,6 @@ asmlinkage void post_ttbr_update_workaround(void) "ic iallu; dsb nsh; isb", ARM64_WORKAROUND_CAVIUM_27456, CONFIG_CAVIUM_ERRATUM_27456)); - - arm64_apply_bp_hardening(); } static int asids_init(void) From patchwork Mon Feb 26 08:20:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129583 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357735lja; Mon, 26 Feb 2018 00:24:51 -0800 (PST) X-Google-Smtp-Source: AH8x224AP3Q4JfTbgaJh937Zw0H6bE0u7KcEGY2HIW8YOjcE2Badg0UdnRFOzybPXim6em8ouq+R X-Received: by 10.98.14.200 with SMTP id 69mr9719269pfo.168.1519633491674; Mon, 26 Feb 2018 00:24:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633491; cv=none; d=google.com; s=arc-20160816; b=RWYiQAZVdrQHWJBVJ/8jMq7XMpXJRJo7SmPP5m83CQiuUD9cVuODkqTkm96IfROQp6 3SJzTyUvNmo8hdWf0NL2CnPxsBkO2kEgUPAGQOvrPyqDlFlHV/oVH83IYSgzWIwFPb35 xrN/WDsJfo69scNYVAjbhwaWm1UgG+66ucu605hf0ptV7WS/8kGlvfHMfBuyJpCKqK/w Ncnqc6Ofm33sYzAoyU+TKIEVFl0giLsxalVyTdLFbdkL5dIk+dElrSdTxlNM4W4cuS9A lQ4s7XMCLBf2Jjm6s5ASdQXyQnnNc+pjijss0850hi5o68XNG9Kz4QHN+CnIxqdRJkgh 7CHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=LDkUfKYFKhXo5brVfmtfx8nxEICwb5REcDsVergX7aE=; b=jEh7KqiLo9dNy5pUgu2rgPJJ1so72f+HfcHVV5UIembQzJUzW+s7UBvUvh2XbTAxZC Pdh5QwgQZwXteb2kptykU8IksGNZfyE3U1gzTHhIhRxm4aAHlmiR7LjLIqV3/1HjLjz8 AVML6YEkS6xffQ3R11ZUzhzW++A3io/WgfzOOxU2DJ5GP1FjlJb0/azv9yn2K7gFCIRy BfUF2KfYrS/dJoRkS/+8BSEwBSARzUp19relgkdQNO35IqiyMTgbF5nmAd08ZedNrQmq WSmQxAOzY7wxm54Kk28g8CT9f+kG9cKhpB9bciTqjMTN68gyLbUGmJMi2e16aS6BCT4x 3Fhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Z+Zttg+b; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 1-v6si6389538plp.678.2018.02.26.00.24.51; Mon, 26 Feb 2018 00:24:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Z+Zttg+b; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752241AbeBZIYs (ORCPT + 28 others); Mon, 26 Feb 2018 03:24:48 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:33339 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751925AbeBZIYe (ORCPT ); Mon, 26 Feb 2018 03:24:34 -0500 Received: by mail-pf0-f194.google.com with SMTP id q13so6223937pff.0 for ; Mon, 26 Feb 2018 00:24:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=LDkUfKYFKhXo5brVfmtfx8nxEICwb5REcDsVergX7aE=; b=Z+Zttg+boqPovLMdmn0u3gkIMcL4DTBIXZ1GnnIg87+EVmNm3t1SfXogG2FwP0G5H3 lL6vmUqc2PP/czCeevJfl1O8HzqwvKMyApJVmjPKaHtYZn+R7A5CGSbBFwwTbwsLL7vp L/rXZfWiVDCAxqOpOqR5a+emsrVxU5O+6gbMU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=LDkUfKYFKhXo5brVfmtfx8nxEICwb5REcDsVergX7aE=; b=krukydZxDhrFAfs0BXRcWZ5hH3BmKoiY5Nh9gbRv8P0fNzvppYa2J1kNIGw1MIS1P9 dMboBc7aszQRWamFCsge5cqGHngdVSQbKdY7jqfXl4lYbMW6UmgYL1g3suYix1hxBJSK 2lTXlT5RdN5MQ6Mpm5+AatBrFJLr5sUwuExOkK/hZzrHLTG99C+YddeuOpmReOnRQv9x e8BJrtlT9SluqjP3cx2tDyQS0HwKLktTqVusFTMa0/ADIeynr2s+idFipEnRgyjPly8F 4q/BTN8CFIWC0B5Hc8Untyy2ASBA45tUlp3hrhGao1u8aFQaRJRLO2Y3tfUz+owTNwdF SWtQ== X-Gm-Message-State: APf1xPCroYSUCFSmDUr6v1bimolGtJCSfm5O5fB3GTCl1lWHDiP+MpxQ dnHjXD4G4KqmbHn4y2XFyQs+Jg== X-Received: by 10.98.144.65 with SMTP id a62mr9769389pfe.96.1519633473513; Mon, 26 Feb 2018 00:24:33 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:32 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 29/52] arm64: Implement branch predictor hardening for affected Cortex-A CPUs Date: Mon, 26 Feb 2018 16:20:03 +0800 Message-Id: <1519633227-29832-30-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Will Deacon commit aa6acde65e03 upstream. Cortex-A57, A72, A73 and A75 are susceptible to branch predictor aliasing and can theoretically be attacked by malicious code. This patch implements a PSCI-based mitigation for these CPUs when available. The call into firmware will invalidate the branch predictor state, preventing any malicious entries from affecting other victim contexts. Co-developed-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi Conflicts: no falkor in arch/arm64/kernel/cpu_errata.c --- arch/arm64/kernel/bpi.S | 24 ++++++++++++++++++++++++ arch/arm64/kernel/cpu_errata.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index 06a931e..dec95bd 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -53,3 +53,27 @@ ENTRY(__bp_harden_hyp_vecs_start) vectors __kvm_hyp_vector .endr ENTRY(__bp_harden_hyp_vecs_end) +ENTRY(__psci_hyp_bp_inval_start) + sub sp, sp, #(8 * 18) + stp x16, x17, [sp, #(16 * 0)] + stp x14, x15, [sp, #(16 * 1)] + stp x12, x13, [sp, #(16 * 2)] + stp x10, x11, [sp, #(16 * 3)] + stp x8, x9, [sp, #(16 * 4)] + stp x6, x7, [sp, #(16 * 5)] + stp x4, x5, [sp, #(16 * 6)] + stp x2, x3, [sp, #(16 * 7)] + stp x0, x1, [sp, #(16 * 8)] + mov x0, #0x84000000 + smc #0 + ldp x16, x17, [sp, #(16 * 0)] + ldp x14, x15, [sp, #(16 * 1)] + ldp x12, x13, [sp, #(16 * 2)] + ldp x10, x11, [sp, #(16 * 3)] + ldp x8, x9, [sp, #(16 * 4)] + ldp x6, x7, [sp, #(16 * 5)] + ldp x4, x5, [sp, #(16 * 6)] + ldp x2, x3, [sp, #(16 * 7)] + ldp x0, x1, [sp, #(16 * 8)] + add sp, sp, #(8 * 18) +ENTRY(__psci_hyp_bp_inval_end) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 0e07893..f8810bf 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -53,6 +53,8 @@ static int cpu_enable_trap_ctr_access(void *__unused) DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); #ifdef CONFIG_KVM +extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; + static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, const char *hyp_vecs_end) { @@ -94,6 +96,9 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn, spin_unlock(&bp_lock); } #else +#define __psci_hyp_bp_inval_start NULL +#define __psci_hyp_bp_inval_end NULL + static void __install_bp_hardening_cb(bp_hardening_cb_t fn, const char *hyp_vecs_start, const char *hyp_vecs_end) @@ -118,6 +123,21 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry, __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end); } + +#include + +static int enable_psci_bp_hardening(void *data) +{ + const struct arm64_cpu_capabilities *entry = data; + + if (psci_ops.get_version) + install_bp_hardening_cb(entry, + (bp_hardening_cb_t)psci_ops.get_version, + __psci_hyp_bp_inval_start, + __psci_hyp_bp_inval_end); + + return 0; +} #endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ #define MIDR_RANGE(model, min, max) \ @@ -211,6 +231,28 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .def_scope = SCOPE_LOCAL_CPU, .enable = cpu_enable_trap_ctr_access, }, +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A73), + .enable = enable_psci_bp_hardening, + }, + { + .capability = ARM64_HARDEN_BRANCH_PREDICTOR, + MIDR_ALL_VERSIONS(MIDR_CORTEX_A75), + .enable = enable_psci_bp_hardening, + }, +#endif { } }; From patchwork Mon Feb 26 08:20:05 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129584 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357856lja; Mon, 26 Feb 2018 00:25:05 -0800 (PST) X-Google-Smtp-Source: AH8x224aZ11a2j4jp7/v58VbyE+HS5VdPh724O90FfTAXQ3uAoHLJwPXkJrSnUuCw2DkTZ7tfs4R X-Received: by 2002:a17:902:6b48:: with SMTP id g8-v6mr9872101plt.151.1519633504951; Mon, 26 Feb 2018 00:25:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633504; cv=none; d=google.com; s=arc-20160816; b=XGZTghTZzgV9gAsULhkKJWoZo2b5mcjz3derObL6ofGjIs5Oket6G7o6Qj7a9vsSfH WgZAwuPDXk42eTP4JheFvj1bRysT3rFc9bTxJHPYzOFuAMrz9aL7wQ8zOvVcuU5Q6cGj BIppZCSYZNwNmdx48qA5Uqn27racINXjmTYxg7g2rkpvX/S7XN+qFFU889K6Spk++Xiv f0i4/bi5L5pDAmHohnpmwn/qsSbwNHKPVKAXEZ+Sy5KjI9bdNaZ9BGnpg4eBPgWsyNW6 QDRFciIPLVGV9W39rSIb3fh1uV2PI1e0TrEtYNWIkNv12jzeuJHI3zhnCWV5S9JcGREV o+WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=KviX1yLvB2tH9OuAASVpED/B9xVdJyxLoOBcpgguMMM=; b=Rx+2c7ksDDCyvuJroHfQG51iwmJpiRhJN0fE4uBys+gJMwPQqQSTj3nHGHd831WkH+ Nn3UdK0UAY8wOVljXbYwGjWhuT4kd+oru+e2W6xvIzcaJMN3lDwoMAhcbvNGlfS2c/7F GfWkbYzi1djQqQwIYlpMFOfpTGoWogPx6qW7yoI5L3qOjXKcatRP9cMJKKQZh6LXqSf+ VoTPHqQ5uonNx41ipB0vUZHJa2fWDQh2blvhTV3Cx5dRx1EIIGbb8FqhHKXRM4s26WdH E0+lRtBzAGapx94KzuE2D1N7WQOsGNEljDXB/sSCOGLlaLiTKmJXE27DJC/MjEl4ss9S 1Hfg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cT9Fkj5j; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u13-v6si6387903plq.823.2018.02.26.00.25.04; Mon, 26 Feb 2018 00:25:04 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cT9Fkj5j; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752643AbeBZIY7 (ORCPT + 28 others); Mon, 26 Feb 2018 03:24:59 -0500 Received: from mail-pl0-f66.google.com ([209.85.160.66]:45820 "EHLO mail-pl0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752024AbeBZIYx (ORCPT ); Mon, 26 Feb 2018 03:24:53 -0500 Received: by mail-pl0-f66.google.com with SMTP id v9-v6so6860116plp.12 for ; Mon, 26 Feb 2018 00:24:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=KviX1yLvB2tH9OuAASVpED/B9xVdJyxLoOBcpgguMMM=; b=cT9Fkj5joEx8GLBL7LoxHOkPuImYad4F2XrfIV64DFL2nls82W1QOzWvDJiwRwGzm9 QoFuxrNRHQcuSeDLiq3z1HIv9KnTNYoeYIhVAdyiCqF7vD6VChIHTCJ8i81/HPfpI109 BVHmLuYfy6jYtKb5MERvW6a2O1/rSoWWzHEV0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=KviX1yLvB2tH9OuAASVpED/B9xVdJyxLoOBcpgguMMM=; b=kDDBbwD9soi2nAI9rG8B31lNIgarArKAU2ETh3bRON2Q+8lOvvDEusJ6+mgCEKFiTp zne6WQCuOS/H6QR3okXHT0v+ifUs8OdPapT4R41NjJTMxGju756K6ozYWt+sDbkQa26G f0eq1PvGv6OdS7nAQeZEBCmSVtOEq15yJ6ODf81uYB20EKX03zSpA0CoH4tNcVB8pmji J+/oLMG9MNjyXpzl3D8ttRn56+uQm6lAlnrEoLYrO7YbJCE5Xz6OTUAsjYmrE+S/XDVj JGXnX8+xiZt9VjPcyQjexeMzk96VstjF4eiEQFt4j+lOfTKHyEAXXf3weqp0ODzO+uzA sWgQ== X-Gm-Message-State: APf1xPDzvZgT4kE/K/7FAfD8tTZ84nxdfMpp7uXuIDClViB/dwz6Pass mNuC4Djf6DvgQdc51Dm/1tAtQw== X-Received: by 2002:a17:902:bf0a:: with SMTP id bi10-v6mr9884384plb.181.1519633492694; Mon, 26 Feb 2018 00:24:52 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:24:52 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 31/52] arm/arm64: KVM: Consolidate the PSCI include files Date: Mon, 26 Feb 2018 16:20:05 +0800 Message-Id: <1519633227-29832-32-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 1a2fb94e6a77 upstream. As we're about to update the PSCI support, and because I'm lazy, let's move the PSCI include file to include/kvm so that both ARM architectures can find it. Acked-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: need kvm/arm_psci.h in files: arch/arm64/kvm/handle_exit.c arch/arm/kvm/psci.c and arch/arm/kvm/arm.c no virt/kvm/arm/arm.c and virt/kvm/arm/psci.c --- arch/arm/include/asm/kvm_psci.h | 27 --------------------------- arch/arm/kvm/arm.c | 2 +- arch/arm/kvm/handle_exit.c | 2 +- arch/arm/kvm/psci.c | 2 +- arch/arm64/include/asm/kvm_psci.h | 27 --------------------------- arch/arm64/kvm/handle_exit.c | 3 ++- include/kvm/arm_psci.h | 27 +++++++++++++++++++++++++++ 7 files changed, 32 insertions(+), 58 deletions(-) delete mode 100644 arch/arm/include/asm/kvm_psci.h delete mode 100644 arch/arm64/include/asm/kvm_psci.h create mode 100644 include/kvm/arm_psci.h -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_psci.h b/arch/arm/include/asm/kvm_psci.h deleted file mode 100644 index 6bda945..0000000 --- a/arch/arm/include/asm/kvm_psci.h +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Copyright (C) 2012 - ARM Ltd - * Author: Marc Zyngier - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#ifndef __ARM_KVM_PSCI_H__ -#define __ARM_KVM_PSCI_H__ - -#define KVM_ARM_PSCI_0_1 1 -#define KVM_ARM_PSCI_0_2 2 - -int kvm_psci_version(struct kvm_vcpu *vcpu); -int kvm_psci_call(struct kvm_vcpu *vcpu); - -#endif /* __ARM_KVM_PSCI_H__ */ diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index 4e4ae07..8588e67 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -44,8 +44,8 @@ #include #include #include -#include #include +#include #ifdef REQUIRES_VIRT __asm__(".arch_extension virt"); diff --git a/arch/arm/kvm/handle_exit.c b/arch/arm/kvm/handle_exit.c index 4e57ebc..0a2b758 100644 --- a/arch/arm/kvm/handle_exit.c +++ b/arch/arm/kvm/handle_exit.c @@ -21,7 +21,7 @@ #include #include #include -#include +#include #include #include "trace.h" diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index a08d7a9..18f9e9d 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -21,8 +21,8 @@ #include #include -#include #include +#include #include diff --git a/arch/arm64/include/asm/kvm_psci.h b/arch/arm64/include/asm/kvm_psci.h deleted file mode 100644 index bc39e55..0000000 --- a/arch/arm64/include/asm/kvm_psci.h +++ /dev/null @@ -1,27 +0,0 @@ -/* - * Copyright (C) 2012,2013 - ARM Ltd - * Author: Marc Zyngier - * - * This program is free software; you can redistribute it and/or modify - * it under the terms of the GNU General Public License version 2 as - * published by the Free Software Foundation. - * - * This program is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with this program. If not, see . - */ - -#ifndef __ARM64_KVM_PSCI_H__ -#define __ARM64_KVM_PSCI_H__ - -#define KVM_ARM_PSCI_0_1 1 -#define KVM_ARM_PSCI_0_2 2 - -int kvm_psci_version(struct kvm_vcpu *vcpu); -int kvm_psci_call(struct kvm_vcpu *vcpu); - -#endif /* __ARM64_KVM_PSCI_H__ */ diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index 5b56b09..0231ebc 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -22,12 +22,13 @@ #include #include +#include + #include #include #include #include #include -#include #define CREATE_TRACE_POINTS #include "trace.h" diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h new file mode 100644 index 0000000..2042bb9 --- /dev/null +++ b/include/kvm/arm_psci.h @@ -0,0 +1,27 @@ +/* + * Copyright (C) 2012,2013 - ARM Ltd + * Author: Marc Zyngier + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#ifndef __KVM_ARM_PSCI_H__ +#define __KVM_ARM_PSCI_H__ + +#define KVM_ARM_PSCI_0_1 1 +#define KVM_ARM_PSCI_0_2 2 + +int kvm_psci_version(struct kvm_vcpu *vcpu); +int kvm_psci_call(struct kvm_vcpu *vcpu); + +#endif /* __KVM_ARM_PSCI_H__ */ From patchwork Mon Feb 26 08:20:06 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129585 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3357928lja; Mon, 26 Feb 2018 00:25:11 -0800 (PST) X-Google-Smtp-Source: AH8x225o+RHGaMEOf1wxmspVIEu/9+lh363GcE+oiQFlMm20AIvKb+Eh0qJjxRsz+5lZIRPNXJXe X-Received: by 10.98.252.22 with SMTP id e22mr9793266pfh.235.1519633511724; Mon, 26 Feb 2018 00:25:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633511; cv=none; d=google.com; s=arc-20160816; b=BOYBuhQ0vhVw0+tOMLdVhScLiydy3xFNRF6sC6EBACxG7V5+tBwhCAvFw2MzQoFYLF z02hnwrDZt8+Pg0i0DNCIq/TfNTnhGD7KiC/6F94kgKjcJV0JIk60yJOuW5zAGZITq1y p5ZMwK5HMvzSP5blNeW0xbWL/uhPjGjxWWUSiQAgkwTSkYdPKKpScVAt8wtpbDkc4USa JM7CHZgwiVu6Yzr806QINLHWbSUsjuwMv7NYveisom/p7+3lzjzBmDuFn5tFzAJ0yPBI JrzC0KVazOJ2Fe0UZsoaigM5S3yn6Bt3zEU+BTtGiSluVQtsbwBKwj9aG3WpkE5jolWZ /3jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=g9Ng8TgnacRSykLoS2YGMqnSYcIU1gnnnP+liPtKoOA=; b=t5oiC3TZYgrCX5LB7F3yYvX7iRvW1rCgeekFEalMqqKRyce6mO8xjIasUx+z3DRpJI XAvjA1kd6wTLf42ZIoLpIMEG3+uI/UW3vyUJ0gQTjYo4gIhpK+9ZHcN+7xZREejZEhjI KmTm/YK8XIQGyxS6qOqLiU7BOwNRZnwmJyDfATJsZedD3rYWYpOa+fuGmN6+vnRpxP1F GhXfsS5BouSk7fbH0iMUXcCl1a70fiP/CIldTnGLkSbVYLm+IMEHaXiQ4hCURRQYO3wx 5w447rbrjPdd4RACjq3vvfrH/NGZYkJt3qv9FuTpSnxoDNfqU1cd9xQeGjl5b8EHqIK1 /vXA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Z3fkc5G6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u13-v6si6387903plq.823.2018.02.26.00.25.11; Mon, 26 Feb 2018 00:25:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Z3fkc5G6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752394AbeBZIZJ (ORCPT + 28 others); Mon, 26 Feb 2018 03:25:09 -0500 Received: from mail-pg0-f66.google.com ([74.125.83.66]:42999 "EHLO mail-pg0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752018AbeBZIZD (ORCPT ); Mon, 26 Feb 2018 03:25:03 -0500 Received: by mail-pg0-f66.google.com with SMTP id y8so5907040pgr.9 for ; Mon, 26 Feb 2018 00:25:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=g9Ng8TgnacRSykLoS2YGMqnSYcIU1gnnnP+liPtKoOA=; b=Z3fkc5G6AaPvHMIyYUJ0hpodFa2KfJZhApW5PIzv5A0E2GlMFUogJMy96bkggOTldY 3zblREGo5hKP1TB2MCe0cXodcbJGeu97Cci3t264kY+2FyUdd/LgPZoOlnQwaTIT+Kx4 1bH5nL05InMhRtQvatSZLf8XZ+MZgspvu2Smc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=g9Ng8TgnacRSykLoS2YGMqnSYcIU1gnnnP+liPtKoOA=; b=k6PIgBlLq9ioShFf0IETNMn5ulOm8NZA20qga/l19OJZFhi0nbuT8VO2Dn9SdBfdBt uJOnL/RuDLoZcajLmeW64gWuqPYSDTU4soKUeBsbViXYRiGY5LOWghUMR4gFbFqNFzon CUPgvE9EspSPMhoP5mfHGaXp+YjeeVE3hKyN3k+BK8xHsex+lgbn29JFvJQSW3WpjlQP vW+vcNLd3nJwqDFyAsRnhiBpuCo8tOxaX/FhLxvG1ag7EzFE8Z6WDHpGYPTlyDWfSiP5 7dDT3qXZytuDwt58wEx7J4iOjQFo5gv6pKZUVO1SSi00H7wG8RnH4aph6CUrqxX6c3/Y ph/A== X-Gm-Message-State: APf1xPDNvywxjNHyYi5X5lNmSaGnqf8gude+JlG92Xs/GelFg/0vdOBX oKqwTC5TUS4+KA1MA00PtaQ7mg== X-Received: by 10.99.4.197 with SMTP id 188mr7600721pge.359.1519633503200; Mon, 26 Feb 2018 00:25:03 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.24.53 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:25:02 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Russell King , Mark Rutland , Lorenzo Pieralisi , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 32/52] arm/arm64: KVM: Add PSCI_VERSION helper Date: Mon, 26 Feb 2018 16:20:06 +0800 Message-Id: <1519633227-29832-33-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit d0a144f12a7c upstream. As we're about to trigger a PSCI version explosion, it doesn't hurt to introduce a PSCI_VERSION helper that is going to be used everywhere. Reviewed-by: Christoffer Dall Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: mv change form virt/kvm/arm/psci.c to arch/arm/kvm/psci.c --- arch/arm/kvm/psci.c | 2 +- include/kvm/arm_psci.h | 6 ++++-- include/uapi/linux/psci.h | 3 +++ 3 files changed, 8 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm/kvm/psci.c b/arch/arm/kvm/psci.c index 18f9e9d..4adfa28 100644 --- a/arch/arm/kvm/psci.c +++ b/arch/arm/kvm/psci.c @@ -219,7 +219,7 @@ static int kvm_psci_0_2_call(struct kvm_vcpu *vcpu) * Bits[31:16] = Major Version = 0 * Bits[15:0] = Minor Version = 2 */ - val = 2; + val = KVM_ARM_PSCI_0_2; break; case PSCI_0_2_FN_CPU_SUSPEND: case PSCI_0_2_FN64_CPU_SUSPEND: diff --git a/include/kvm/arm_psci.h b/include/kvm/arm_psci.h index 2042bb9..5659343 100644 --- a/include/kvm/arm_psci.h +++ b/include/kvm/arm_psci.h @@ -18,8 +18,10 @@ #ifndef __KVM_ARM_PSCI_H__ #define __KVM_ARM_PSCI_H__ -#define KVM_ARM_PSCI_0_1 1 -#define KVM_ARM_PSCI_0_2 2 +#include + +#define KVM_ARM_PSCI_0_1 PSCI_VERSION(0, 1) +#define KVM_ARM_PSCI_0_2 PSCI_VERSION(0, 2) int kvm_psci_version(struct kvm_vcpu *vcpu); int kvm_psci_call(struct kvm_vcpu *vcpu); diff --git a/include/uapi/linux/psci.h b/include/uapi/linux/psci.h index 3d7a0fc..39930ca 100644 --- a/include/uapi/linux/psci.h +++ b/include/uapi/linux/psci.h @@ -87,6 +87,9 @@ (((ver) & PSCI_VERSION_MAJOR_MASK) >> PSCI_VERSION_MAJOR_SHIFT) #define PSCI_VERSION_MINOR(ver) \ ((ver) & PSCI_VERSION_MINOR_MASK) +#define PSCI_VERSION(maj, min) \ + ((((maj) << PSCI_VERSION_MAJOR_SHIFT) & PSCI_VERSION_MAJOR_MASK) | \ + ((min) & PSCI_VERSION_MINOR_MASK)) /* PSCI features decoding (>=1.0) */ #define PSCI_1_0_FEATURES_CPU_SUSPEND_PF_SHIFT 1 From patchwork Mon Feb 26 08:20:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129589 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358509lja; Mon, 26 Feb 2018 00:25:54 -0800 (PST) X-Google-Smtp-Source: AH8x224MORAkvqvO50lLxyuRz8/EjWuBLnhyvds5cwM3mSUydXpcQr6/dSlYd25w9JHULd4cwZYM X-Received: by 10.99.116.25 with SMTP id p25mr7910810pgc.109.1519633554393; Mon, 26 Feb 2018 00:25:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633554; cv=none; d=google.com; s=arc-20160816; b=Am+lAZ/IKmM2rXuOZd1L5LUPyS357hS0EsPn9Ggbwlew8oLv7VGG4EoYSxuELzOW/Q vsX5jzeg87b+NPQLVR5b/FMgV14C5602bVJmRljVna1cItcX8dUB/MlMGP1TSH3IHWBN fqhgZTQcH2QF96eFeSccIDyZhoAIUCKFmS6lSHC66IDIksVjt7RRUwJRp77uXqSEr0sq Yg48hKtH5csm1+jWOX8MHY1tpH0DfPHFSc1ryGtrz6WyQfP+jnbP6MoR/BBx1j2TAdyl BGS7Ba6uj8PUbx05TOWhdytct/UTzFpiGnJ4vfdHB4tILUjDSp93YBaymSm3Fy0hIWQU 0Qmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=ccWpB6B5MEm475gq8sjuImnJLMpKY94QLsRYdXyhfpw=; b=yrOSN8iHO1ojCEIbLiQwui3VYmzwDacdac3vN0Uk1tms73Du2GCxRQIIaSUvjHX3rL WnSs3pCbqzaWyACcCfAhJ5ZyiXXVgDRVuKl7VVnPdxqgLj8BH4NSJFv395bplS2hbe9d FPYN0w3vvN4M3BLH+bQaTyvo/7bOjtcxRxgw9fDjGdm/xG+iwAcpHcBfY4F552oh9LAP FNH3POKvcvgI6Z0HGsHSup24NsEKip+s1jXbcbw5fvBfi+FaTe2qZLuWB5f+Xt2HKc21 R0eaRhC4NVTEdkOf7ESz/RgTvddbRZULtJxwc5V5Ota2uq9H84uNANd/3veAsljWEkOg TDLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GMSh1cUz; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u62si5274966pgd.579.2018.02.26.00.25.54; Mon, 26 Feb 2018 00:25:54 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GMSh1cUz; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752057AbeBZIZv (ORCPT + 28 others); Mon, 26 Feb 2018 03:25:51 -0500 Received: from mail-pf0-f193.google.com ([209.85.192.193]:46945 "EHLO mail-pf0-f193.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752102AbeBZIZo (ORCPT ); Mon, 26 Feb 2018 03:25:44 -0500 Received: by mail-pf0-f193.google.com with SMTP id z10so1883406pfh.13 for ; Mon, 26 Feb 2018 00:25:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=ccWpB6B5MEm475gq8sjuImnJLMpKY94QLsRYdXyhfpw=; b=GMSh1cUzLcfYYCAjsxCGkkHOlpOgzYDYRr0AeSZcqTDL7AidefGBvuSZCvhK7rVp8R 7INieTcJI8C6FRncoiPVFIRmwm05Dv79AbD0SNdNLvmqSUI5Ec0I4dq1lzD9/t9Wxr1Q rwz+LMTWWjfuIhgCB4P7HEeg/pbSH8bb0pNuE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=ccWpB6B5MEm475gq8sjuImnJLMpKY94QLsRYdXyhfpw=; b=qJNyK/1FDb/Mjj6FDvt8PgFz+FBm8hcAWNPTOx+ojEJA/Zvyd3IaF9FEx9R0kB3yVA FbUlP24enNy+yyQUA9FWX0F2zPHr3K8BfDYXdsdaue0oOkIsFQ9dFg5RWfDCkT6VQ1iY Gf8qpykp2HpJNLbJVcTF597s9XRm3WW9yGw/yJwf26mlfwWPv508i0zkB53B2ujMbeBL S/S5W3fSDkrz1w/zAXqKHoTdV8nrq+Yz5f0CCL9mj7lVCncn2aSEn0s3kdXwBlfNVG6z V6tr/ali8kpc8bnCykRmQiAN1hjBcHSWy3PHEk9TlRJ9a1VfvLQtddoz1ZVDagfc0D7w CKgg== X-Gm-Message-State: APf1xPCJnraWiMor/+z4ja0w9DAv4+vSGALSdDzp5dzVqlCG6AoqHMWo dC8u2pReQBKiI1xZkstOy1xa4Q== X-Received: by 10.99.167.66 with SMTP id w2mr642864pgo.357.1519633543999; Mon, 26 Feb 2018 00:25:43 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.25.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:25:43 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Christoffer Dall , kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 36/52] arm64: KVM: Make PSCI_VERSION a fast path Date: Mon, 26 Feb 2018 16:20:10 +0800 Message-Id: <1519633227-29832-37-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 90348689d500 upstream. For those CPUs that require PSCI to perform a BP invalidation, going all the way to the PSCI code for not much is a waste of precious cycles. Let's terminate that call as early as possible. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Catalin Marinas Signed-off-by: Alex Shi --- arch/arm64/kvm/hyp/switch.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.7.4 diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index cf6d962..3eab6ac 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -17,6 +17,7 @@ #include #include +#include #include #include @@ -308,6 +309,18 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu)) goto again; + if (exit_code == ARM_EXCEPTION_TRAP && + (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 || + kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32) && + vcpu_get_reg(vcpu, 0) == PSCI_0_2_FN_PSCI_VERSION) { + u64 val = PSCI_RET_NOT_SUPPORTED; + if (test_bit(KVM_ARM_VCPU_PSCI_0_2, vcpu->arch.features)) + val = 2; + + vcpu_set_reg(vcpu, 0, val); + goto again; + } + if (static_branch_unlikely(&vgic_v2_cpuif_trap) && exit_code == ARM_EXCEPTION_TRAP) { bool valid; From patchwork Mon Feb 26 08:20:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129605 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3361476lja; Mon, 26 Feb 2018 00:30:15 -0800 (PST) X-Google-Smtp-Source: AH8x225aLOANYlIu3DXePd2dvlOgdsimffC4k0N8aTOyf4LP9QXZ0EwDWWM7JuGxbIgHVu1LqjLl X-Received: by 10.99.103.195 with SMTP id b186mr7818540pgc.446.1519633815179; Mon, 26 Feb 2018 00:30:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633815; cv=none; d=google.com; s=arc-20160816; b=n3BQWdKwAWbJOiA+JP6+YGO2d1cnTFNMD46q0r3Of1TkTsmLEZmqHvEgue/+Xou66R nYEGiydhqhJGj/YiJtQsPt/50M7EXDazjsEse0kiOHipU8T0TLFWtsy5RRdQ1LB6h2NX DzL+/m/FIF6vYIQ0ZrgR4lykLDwGbEt9Buf3pf7oIKejAYVQjBPJ43RG72N6rQBivUbk j02bsAGc8u2vjcOZtf4Fb3pzBNmsBfxlnW7A3g4Fc55vfrgruC1FcdxhTjI5iSh+/Sh9 zHwCfC0bsJ0eg2jHRGr8+b86ajBTPi4kM5VTMTT1vTeZXPtDPDoIKTx77u2dUHQDCJew v2NA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=Jl8J60UuHWw6QycIueIONGAI64GFje0f/SkXAPYopdo=; b=Q4srWKhQmzakL386X3i/07Ryr3OxnebsQneBfbWxx7tmBanpHEbbNTafmqR0P4agwm GYok/eaLEbZhGitP0MTHCM/BfVn5XCwPuN2hWDeZz3UVKC2jmzx7R8fW16YWbInqEpQm W78MW2yPUIlyXwywsdgN83FW7N7TEqTtv4cWRDndlqxgNPdkvkANC/QMe7MCmqtoZqge LPIuXyaFaMdoX22fyflzvPDyu9LxqPL6qHgtYnnaBlNeJKQDppkwrz/MK4wByYqIRcXk RbpwewYsIn2yu6Uw2XMunHvxBqc3jxwVlMNx/Vh4QSbnjGaugR5gvl0RTKbw5VDoZ4qX aWRA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CYKbZ8pn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i130si1118048pgc.460.2018.02.26.00.30.14; Mon, 26 Feb 2018 00:30:15 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CYKbZ8pn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752731AbeBZI0Y (ORCPT + 28 others); Mon, 26 Feb 2018 03:26:24 -0500 Received: from mail-pg0-f68.google.com ([74.125.83.68]:32913 "EHLO mail-pg0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752391AbeBZI0O (ORCPT ); Mon, 26 Feb 2018 03:26:14 -0500 Received: by mail-pg0-f68.google.com with SMTP id g12so5913663pgs.0 for ; Mon, 26 Feb 2018 00:26:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=Jl8J60UuHWw6QycIueIONGAI64GFje0f/SkXAPYopdo=; b=CYKbZ8pnDKagzMCcokfegFOCOhRYPmf2YAjOPU029FgdAmQX73CpGyxaer6cII0oaS voEFKNYHIDz87nM5+wC7Gxn/nncRUADnG1v+jkzMK5FplMC2Q62zdZUhN65W5J3u89pA 5gop4xM+s89hQGsTahh2hI7y/QvemIodSnaoE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Jl8J60UuHWw6QycIueIONGAI64GFje0f/SkXAPYopdo=; b=pOBnsCW1dLcDTqq//nvtREVa9PnisjvCSGfKrdUuTZfhL9HwX9wqSxzT2iqSC0WmsX n5XpbiTf6EKg7t8oj7FfLgtBvWhdOSH7vCUUo955OtXdCGRph9BF012pj3+07Y8h3478 iKD+13f4hZvZryDid583KRdomMOXv8wRVkKHTGQOL8dFQRep5fpnc22Ymju3ggHKMuFg V8SsRRPgY9hNXkRn+kaa324SeLWtfyEF23yboVKNhI4/0ECn+XXsZsl+q1D9P+6wLHgJ A1O1CK8O9+6RT5HgjQ3bENn0NpcgF3O1yfTpJgKTR82MWOokj7cPRUtZSAb2QapAv4YU kqpA== X-Gm-Message-State: APf1xPAKU3gOpRrXnKZcuIik5I+sr/AjDbyOu+Z5xsodw37vjNJTqp/e aqN1Pld3HOXOfkQL4ewqNdaOYg== X-Received: by 10.99.121.203 with SMTP id u194mr8076290pgc.232.1519633573562; Mon, 26 Feb 2018 00:26:13 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:13 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 39/52] arm64: KVM: Add SMCCC_ARCH_WORKAROUND_1 fast handling Date: Mon, 26 Feb 2018 16:20:13 +0800 Message-Id: <1519633227-29832-40-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit f72af90c3783 upstream. We want SMCCC_ARCH_WORKAROUND_1 to be fast. As fast as possible. So let's intercept it as early as we can by testing for the function call number as soon as we've identified a HVC call coming from the guest. Tested-by: Ard Biesheuvel Reviewed-by: Christoffer Dall Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm64/kvm/hyp/hyp-entry.S | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kvm/hyp/hyp-entry.S b/arch/arm64/kvm/hyp/hyp-entry.S index 4e92399..4e9d50c 100644 --- a/arch/arm64/kvm/hyp/hyp-entry.S +++ b/arch/arm64/kvm/hyp/hyp-entry.S @@ -15,6 +15,7 @@ * along with this program. If not, see . */ +#include #include #include @@ -79,10 +80,11 @@ alternative_endif lsr x0, x1, #ESR_ELx_EC_SHIFT cmp x0, #ESR_ELx_EC_HVC64 + ccmp x0, #ESR_ELx_EC_HVC32, #4, ne b.ne el1_trap - mrs x1, vttbr_el2 // If vttbr is valid, the 64bit guest - cbnz x1, el1_trap // called HVC + mrs x1, vttbr_el2 // If vttbr is valid, the guest + cbnz x1, el1_hvc_guest // called HVC /* Here, we're pretty sure the host called HVC. */ ldp x0, x1, [sp], #16 @@ -101,6 +103,20 @@ alternative_endif 2: eret +el1_hvc_guest: + /* + * Fastest possible path for ARM_SMCCC_ARCH_WORKAROUND_1. + * The workaround has already been applied on the host, + * so let's quickly get back to the guest. We don't bother + * restoring x1, as it can be clobbered anyway. + */ + ldr x1, [sp] // Guest's x0 + eor w1, w1, #ARM_SMCCC_ARCH_WORKAROUND_1 + cbnz w1, el1_trap + mov x0, x1 + add sp, sp, #16 + eret + el1_trap: /* * x0: ESR_EC From patchwork Mon Feb 26 08:20:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129592 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3358973lja; Mon, 26 Feb 2018 00:26:32 -0800 (PST) X-Google-Smtp-Source: AH8x224Pt5x4McBBcY43LgKz7XI6vmgI0zk9AxRx9xS0uz2Le73IH+Ghu8XaHVGQgDhj7ANoRLON X-Received: by 2002:a17:902:7c11:: with SMTP id x17-v6mr9978446pll.59.1519633592546; Mon, 26 Feb 2018 00:26:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633592; cv=none; d=google.com; s=arc-20160816; b=y64HNw5iIHrx4kevlGA6GbXqdQQOoxaU0U7K5cU8WUSRsBHWNYXfpbjMcjuY4JC4TR jE4PLmY76KJLDJ5mBlh3C4UZuAPPCv5OWt0aBTGv5mPiFUsL6yEzrnfVBE6O5CaQZQ+D dRk2fPGsx2IpxMxfeUvYpw+QDqEx+jdqWhJWipGZ4fFo8vd2WzZQlCKaioEQ6icdhbR5 TabRIJiO3h9hSxdGo+O6trTX69UUy/xkJZBkxUyI3XhQJrJ849JO7XkESkzDlTI/RiqD F8MTYG5Fo3JYTSGs1PyNbVaSSETBhAEhXgGmqLSXpesPPTYCYVUrEkz4n7Fvd4yJGsQu BNMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=JR0dWBMHXzRrhmbkXPj+t5C5z62Z2EIDM9UumhZBlIo=; b=Qm210QJuFHqjPuX+iWCgb/h+u5I8X1I/zEsPf+RDTJq/2W+VCO0I1/SPbLzJml2r1z D0rZibu9lH4lE2tIC8177Bezq9fHcrDWIhEfs30ZAsncFJrSyB1OvgXJhjiOtBDq46S/ LSU6CrYbPi+lWI10NT4pYwsBpMJY4Xd/EdLPEPvgJ+dMFw9i1AL32Llh+rOWMsuT4Qkd sRbh1jP/VDS5hedMdNMxTo/BuW0bfnLJ0R0LyscyveRt9IRCmVIzW/BrPy5ofNsnabzw TAM79ZrFb7zfh1VStNIlq2dpsArrJgjfkpIALo4J1o7tGEzzhKYTzDxV4yW3ZluHnsI+ ZCkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=huHrpEo6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l14si5214459pgc.615.2018.02.26.00.26.32; Mon, 26 Feb 2018 00:26:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=huHrpEo6; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752747AbeBZI03 (ORCPT + 28 others); Mon, 26 Feb 2018 03:26:29 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:41326 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752567AbeBZI0V (ORCPT ); Mon, 26 Feb 2018 03:26:21 -0500 Received: by mail-pf0-f196.google.com with SMTP id f80so1122149pfa.8 for ; Mon, 26 Feb 2018 00:26:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=JR0dWBMHXzRrhmbkXPj+t5C5z62Z2EIDM9UumhZBlIo=; b=huHrpEo6HLIOd/KCc5P5goXLlPyVPKyl03CHY/finVtxl91PHNnKqPl1/1YP2WqAsm NjdQMb7k5FS24kD+aFBkl9E5Mj/OfUQUs32hPUlH3OeRuMeUITA0wu0ISkHiNuH9Gevb LGTstrvHqEnLSBwKPgo7TG+lM+9ekuPdvnG34= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=JR0dWBMHXzRrhmbkXPj+t5C5z62Z2EIDM9UumhZBlIo=; b=mlKD1HasmXV+iqWfbqTaYvsmBP6+S6jjrVZowh8/3C9MhhW5feE7RzfQ1LPq3JuL7/ ohxFy+Mtl5zu5JwHKnxHOfoKHsud5FMvsn4r2OW3wH08J2TvIYIxydBo/I76QuHKXfYs hGlqrIy6BRdWFr4jNnW6MdshNeWOifdjaPmx0gW9zNyodsgjf5jnujDrtSETnT0xFaSt vMVX1xXbJBzYjk4ltx92c1jjKCohCdmbxpS1u9TF9cR02EBAywHuLVlCLzE50yG1EO4s v5C0mdRp5YQ3VJCAgFPeyOyT/SbY6hAkUU9P6dd9F/gQItGm/mqS22ZLiMfWv2poy4nI 73pQ== X-Gm-Message-State: APf1xPCGSqgKjeQM/o1ICQSIF0vwswc4aOZjFgDi04Pyal8Ma3uoFkoO 3KarEJkZjrBu+4k/MB4dyGz/IQ== X-Received: by 10.167.131.135 with SMTP id u7mr9827489pfm.50.1519633581094; Mon, 26 Feb 2018 00:26:21 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:20 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Mark Rutland , Lorenzo Pieralisi , linux-arm-kernel@lists.infradead.org (open list:POWER STATE COORDINATION INTERFACE (PSCI)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 40/52] firmware/psci: Expose PSCI conduit Date: Mon, 26 Feb 2018 16:20:14 +0800 Message-Id: <1519633227-29832-41-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 09a8d6d48499 upstream. In order to call into the firmware to apply workarounds, it is useful to find out whether we're using HVC or SMC. Let's expose this through the psci_ops. Acked-by: Lorenzo Pieralisi Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- drivers/firmware/psci.c | 28 +++++++++++++++++++++++----- include/linux/psci.h | 7 +++++++ 2 files changed, 30 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c index 9a3ce76..a49196a 100644 --- a/drivers/firmware/psci.c +++ b/drivers/firmware/psci.c @@ -59,7 +59,9 @@ bool psci_tos_resident_on(int cpu) return cpu == resident_cpu; } -struct psci_operations psci_ops; +struct psci_operations psci_ops = { + .conduit = PSCI_CONDUIT_NONE, +}; typedef unsigned long (psci_fn)(unsigned long, unsigned long, unsigned long, unsigned long); @@ -210,6 +212,22 @@ static unsigned long psci_migrate_info_up_cpu(void) 0, 0, 0); } +static void set_conduit(enum psci_conduit conduit) +{ + switch (conduit) { + case PSCI_CONDUIT_HVC: + invoke_psci_fn = __invoke_psci_fn_hvc; + break; + case PSCI_CONDUIT_SMC: + invoke_psci_fn = __invoke_psci_fn_smc; + break; + default: + WARN(1, "Unexpected PSCI conduit %d\n", conduit); + } + + psci_ops.conduit = conduit; +} + static int get_set_conduit_method(struct device_node *np) { const char *method; @@ -222,9 +240,9 @@ static int get_set_conduit_method(struct device_node *np) } if (!strcmp("hvc", method)) { - invoke_psci_fn = __invoke_psci_fn_hvc; + set_conduit(PSCI_CONDUIT_HVC); } else if (!strcmp("smc", method)) { - invoke_psci_fn = __invoke_psci_fn_smc; + set_conduit(PSCI_CONDUIT_SMC); } else { pr_warn("invalid \"method\" property: %s\n", method); return -EINVAL; @@ -654,9 +672,9 @@ int __init psci_acpi_init(void) pr_info("probing for conduit method from ACPI.\n"); if (acpi_psci_use_hvc()) - invoke_psci_fn = __invoke_psci_fn_hvc; + set_conduit(PSCI_CONDUIT_HVC); else - invoke_psci_fn = __invoke_psci_fn_smc; + set_conduit(PSCI_CONDUIT_SMC); return psci_probe(); } diff --git a/include/linux/psci.h b/include/linux/psci.h index 6306ab1..66ff547 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -25,6 +25,12 @@ bool psci_tos_resident_on(int cpu); int psci_cpu_init_idle(unsigned int cpu); int psci_cpu_suspend_enter(unsigned long index); +enum psci_conduit { + PSCI_CONDUIT_NONE, + PSCI_CONDUIT_SMC, + PSCI_CONDUIT_HVC, +}; + struct psci_operations { u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); @@ -34,6 +40,7 @@ struct psci_operations { int (*affinity_info)(unsigned long target_affinity, unsigned long lowest_affinity_level); int (*migrate_info_type)(void); + enum psci_conduit conduit; }; extern struct psci_operations psci_ops; From patchwork Mon Feb 26 08:20:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129593 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359067lja; Mon, 26 Feb 2018 00:26:41 -0800 (PST) X-Google-Smtp-Source: AH8x227NlVOD9fB8QSGp2+Qa9+xlM47Iz94CwMZFY/eJXjEM6fDZoXqzeIAwkXesFOIHBU6DY3xk X-Received: by 2002:a17:902:aa5:: with SMTP id 34-v6mr9997049plp.429.1519633601001; Mon, 26 Feb 2018 00:26:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633600; cv=none; d=google.com; s=arc-20160816; b=LrIIa7EFEazazbIzPhaCT/341mR6wCKdqKzfc28xSSu7iR6EWL/dGesBf8AOMHygyv hXpqJepXUUAnMKQ6/icrZQveSyRfCJOA1EYJ9of+6xwu5mFnAcIR5UvC2ziJzbGJ5ix1 x6JjoBNlt5LVnaUTan/e7LZWItK7jrntlWQXz1YxIn2lX7yIF4dUoSRAykDpI2f1mhpT seIu4G5CUSebVOB8MHKjYyPr/7dqjnzth3aZ4d77xZOSQe6yFEw7o0aUYeiOZGzwPEco Asl3djP9zNpvXzZ3z8dq/fUaEDvcGVM8/O1rkHj71AhGI0ul7tk0UCErZjAELXuNUoCe Cn2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=HEY4UErd4E33sLr2mfE4fjyOkfyGVz0DVWFLdDXHqoI=; b=j1BnvQ0hBytA/2nSeRonzFBlZKHBS/29gQNxhNpVw2DIdrKuHj06ZQgHdOE86z8TF5 MYjbs8eH1WLwpo90Pqaq/GCSDdVs0AvcMNVePkaRW/Q6uO4VLr7FR3fISSaqa2t6qpzM Oc1eOUVHdeSH1X+5HoMQ+SaIrIUKpMS58Sx/d3mUafl/xfHQSOZkIFEeGJfKqyeSM58A bDZ1EpZt+X5MrVy8zu9GyttxB7oleRboCz7NTPVIX23e2XbwRFy8ErAmwvycFI1sASbu Nu216CDn4wDIjvdF5EAJKaSuD19fDPzCv9t/JBM/9wRjoykFxEUxOQmXTQFfdD8aNYUF ZJog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OLo4f6AI; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a22-v6si5233862plm.773.2018.02.26.00.26.40; Mon, 26 Feb 2018 00:26:40 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OLo4f6AI; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752759AbeBZI0h (ORCPT + 28 others); Mon, 26 Feb 2018 03:26:37 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:35366 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752749AbeBZI0a (ORCPT ); Mon, 26 Feb 2018 03:26:30 -0500 Received: by mail-pf0-f194.google.com with SMTP id y186so5539910pfb.2 for ; Mon, 26 Feb 2018 00:26:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=HEY4UErd4E33sLr2mfE4fjyOkfyGVz0DVWFLdDXHqoI=; b=OLo4f6AIJid1kE3t0Y+Wej9HWk22PABGTv5XIJNAZqU98RIJgtNa4CYF2c+HNsNzJB qlrchuxEBU/QJp/6Wz3X/2DXKcq3CRBo1yLgGQsk5dbMvQs7UEYPyAC/YV3N9GNBCqIX IeDWsDSrvhWJIHrywoQVodGECoLJCawx/zu5s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=HEY4UErd4E33sLr2mfE4fjyOkfyGVz0DVWFLdDXHqoI=; b=b4wQjNnCWfENtHtumiE11DIpAot4z96rzklak3oZ67V3KBvHrsNTG7Lvk0tND8zF/r rNfSObb6tk/zp0uOAtCjS39+KIOtr9tDnzIt7qCuYPjsi9x9norEq9o3Nz9D9DhbtSX5 1xvtnB6hUSIo+wyrTPCv9XiItNMxJIl4gQa9T/SbvO1+oecZECOFDhPQPXq3xM6Pzb7P Ln7Rh8Ewt9Ly7x0yEa7iqhOFAwo+YFku31MCw2+nA9dzluMbVUdYkTCerjb6umRAtnZW KrdC95TJ9pQ6o842HaqqdwtTClf/iBouhjr/A90dRDIoX6ZNRQRvb1j3536L3dyjU3FY 1O7g== X-Gm-Message-State: APf1xPAyc6UZ7/J3thmn/jlwTkmuwDRsI9irc6P7LSpa4HI9FrzxkuhR XRvGUjcDELnUqFwUgsSnpCy5IA== X-Received: by 10.99.150.2 with SMTP id c2mr7979547pge.424.1519633589367; Mon, 26 Feb 2018 00:26:29 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:28 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Mark Rutland , Lorenzo Pieralisi , linux-arm-kernel@lists.infradead.org (open list:POWER STATE COORDINATION INTERFACE (PSCI)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 41/52] firmware/psci: Expose SMCCC version through psci_ops Date: Mon, 26 Feb 2018 16:20:15 +0800 Message-Id: <1519633227-29832-42-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit e78eef554a91 upstream. Since PSCI 1.0 allows the SMCCC version to be (indirectly) probed, let's do that at boot time, and expose the version of the calling convention as part of the psci_ops structure. Acked-by: Lorenzo Pieralisi Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- drivers/firmware/psci.c | 27 +++++++++++++++++++++++++++ include/linux/psci.h | 6 ++++++ 2 files changed, 33 insertions(+) -- 2.7.4 diff --git a/drivers/firmware/psci.c b/drivers/firmware/psci.c index a49196a..79a48c3 100644 --- a/drivers/firmware/psci.c +++ b/drivers/firmware/psci.c @@ -61,6 +61,7 @@ bool psci_tos_resident_on(int cpu) struct psci_operations psci_ops = { .conduit = PSCI_CONDUIT_NONE, + .smccc_version = SMCCC_VERSION_1_0, }; typedef unsigned long (psci_fn)(unsigned long, unsigned long, @@ -511,6 +512,31 @@ static void __init psci_init_migrate(void) pr_info("Trusted OS resident on physical CPU 0x%lx\n", cpuid); } +static void __init psci_init_smccc(void) +{ + u32 ver = ARM_SMCCC_VERSION_1_0; + int feature; + + feature = psci_features(ARM_SMCCC_VERSION_FUNC_ID); + + if (feature != PSCI_RET_NOT_SUPPORTED) { + u32 ret; + ret = invoke_psci_fn(ARM_SMCCC_VERSION_FUNC_ID, 0, 0, 0); + if (ret == ARM_SMCCC_VERSION_1_1) { + psci_ops.smccc_version = SMCCC_VERSION_1_1; + ver = ret; + } + } + + /* + * Conveniently, the SMCCC and PSCI versions are encoded the + * same way. No, this isn't accidental. + */ + pr_info("SMC Calling Convention v%d.%d\n", + PSCI_VERSION_MAJOR(ver), PSCI_VERSION_MINOR(ver)); + +} + static void __init psci_0_2_set_functions(void) { pr_info("Using standard PSCI v0.2 function IDs\n"); @@ -559,6 +585,7 @@ static int __init psci_probe(void) psci_init_migrate(); if (PSCI_VERSION_MAJOR(ver) >= 1) { + psci_init_smccc(); psci_init_cpu_suspend(); psci_init_system_suspend(); } diff --git a/include/linux/psci.h b/include/linux/psci.h index 66ff547..347077c 100644 --- a/include/linux/psci.h +++ b/include/linux/psci.h @@ -31,6 +31,11 @@ enum psci_conduit { PSCI_CONDUIT_HVC, }; +enum smccc_version { + SMCCC_VERSION_1_0, + SMCCC_VERSION_1_1, +}; + struct psci_operations { u32 (*get_version)(void); int (*cpu_suspend)(u32 state, unsigned long entry_point); @@ -41,6 +46,7 @@ struct psci_operations { unsigned long lowest_affinity_level); int (*migrate_info_type)(void); enum psci_conduit conduit; + enum smccc_version smccc_version; }; extern struct psci_operations psci_ops; From patchwork Mon Feb 26 08:20:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129594 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359253lja; Mon, 26 Feb 2018 00:26:55 -0800 (PST) X-Google-Smtp-Source: AH8x225SIlYkdVhP5+/1MVij0gxUjEDfrCh2ViaRLQZUk71VZju6CzInaKrjAKnZZoppzbABq3C0 X-Received: by 10.99.109.138 with SMTP id i132mr7973118pgc.417.1519633615470; Mon, 26 Feb 2018 00:26:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633615; cv=none; d=google.com; s=arc-20160816; b=lQtR0Z1lqcrjhGdiFcaJbO7FOTZdQV80A1aKeZ89xMj/t7OQ1AXNoP6W5sTRV76Yys SXMJuT5JexT79F0gyED3/e9PnKcAwu1gk5EWSLJViVBsAPYVJM2GE+HOnwA/yt+C6n2B 8zurchE2pnj31ocIG3mVa8PcLtJ2N1JPYB6IEIy3paQ1GIu+RM09dwhjc+V30Av1mHhU mACRjxB2mdOQUh6VSv2OavkP+6kJFTlRH/gTp5FNYUR4dXBFjvZcOAFnNfZB2LaoSsrT 4a06Gu7rliUiZiaSrXsCowXBeF2dR5kLankcmZYT2kr1RoedjlS8UkLEEDwracBAEwcf C60g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=xjpJ8k7CKNbvTvoCWi2+8tjs1SrG8L5eeUCBFmDTG3o=; b=WHK6GFqAJCJHrz29cOp/qdx9+bZNdA1eMyxK8EdMBbx53ergRZSqvlt0Rqhob2ZRs1 Zzlfdn+N/lymQdBtzyJx9qDrqNwLvRO2RKoA0wZBUnIPdcjPb3JsuzX7qtIj0nSgQRzv h4YEZog8QCsPkWanY142DgyTfAnMyC2wdBAKe3rhuBM5jIsPYRo1bqHBSTpEhzTRU9KO AKS0M07tyuHZ72RtYeFDyLZIjhBa9uB8FJJTEgTLZUSzNiRnUr/08Q8+godgRiX9dusb Xbh09vHqcO8omMtuGxn0zHsq9NySHDT5skcVKpDBD7v2sDpOEumFXi9NudpAeBEG8kR0 Bh8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eolpxWlB; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a22-v6si5233862plm.773.2018.02.26.00.26.55; Mon, 26 Feb 2018 00:26:55 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=eolpxWlB; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752786AbeBZI0v (ORCPT + 28 others); Mon, 26 Feb 2018 03:26:51 -0500 Received: from mail-pl0-f68.google.com ([209.85.160.68]:36131 "EHLO mail-pl0-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752492AbeBZI0h (ORCPT ); Mon, 26 Feb 2018 03:26:37 -0500 Received: by mail-pl0-f68.google.com with SMTP id v3so8876818plg.3 for ; Mon, 26 Feb 2018 00:26:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=xjpJ8k7CKNbvTvoCWi2+8tjs1SrG8L5eeUCBFmDTG3o=; b=eolpxWlBa75t/zeVVNzb1Lw41VpQDpfw0BYbrrKAemh0rpgFigirHIYmN5kgW+gJ8p QN2zViHI9mYh5JJub6Z5mFGUi1rblwUNCvhYB624xbndCuL7iu72Ak/GfBr5iHlsYRz2 riuQ7V8VH4wH9/WzMjTZ+i7K5k4txubAZD6ro= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=xjpJ8k7CKNbvTvoCWi2+8tjs1SrG8L5eeUCBFmDTG3o=; b=GCwLWo5xqX66n+9rzbyOortunaTIk8VuE/j2nyAmbSqci8N89ACEtyZtEic58HeMHu 6TkZoAH1MTgfkISUCn7l7M7Di9JoDIKSWJYPZmIrm0hCTzbAqoV0o5Qf4yDpa9IniY2E 2BwbN3LoI/nqNVLweF6c+SZ0lxZjuUDgaqUVJdEygkcXvH9nMs3ufbRo3I2xFQrqh44p ABUz0qFhTys5NASXDZjynE3ON4h3JHLICnF4PnIZXiquZ9N2Wgb0TjW7ANJJlNlGwuxH YrKzHu2s7VTZHFxq2ixYnrkUmpyBkURipCcbBphiEd5PO4Z3BebG9JPnvD9Bcil6DTwZ P1Ng== X-Gm-Message-State: APf1xPB0975VraHqbpDihY7RfQYkmyxWGmoSZziGLBZfHK1HOQDVYYBs tvqFhE7JeiS9WXOZBaIAo7WI3w== X-Received: by 2002:a17:902:5e3:: with SMTP id f90-v6mr9951543plf.413.1519633595873; Mon, 26 Feb 2018 00:26:35 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:35 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-kernel@vger.kernel.org (open list) Subject: [PATCH 42/52] arm/arm64: smccc: Make function identifiers an unsigned quantity Date: Mon, 26 Feb 2018 16:20:16 +0800 Message-Id: <1519633227-29832-43-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit ded4c39e93f3 upstream. Function identifiers are a 32bit, unsigned quantity. But we never tell so to the compiler, resulting in the following: 4ac: b26187e0 mov x0, #0xffffffff80000001 We thus rely on the firmware narrowing it for us, which is not always a reasonable expectation. Cc: stable@vger.kernel.org Reported-by: Ard Biesheuvel Acked-by: Ard Biesheuvel Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- include/linux/arm-smccc.h | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index e1ef944..dd44d84 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -14,14 +14,16 @@ #ifndef __LINUX_ARM_SMCCC_H #define __LINUX_ARM_SMCCC_H +#include + /* * This file provides common defines for ARM SMC Calling Convention as * specified in * http://infocenter.arm.com/help/topic/com.arm.doc.den0028a/index.html */ -#define ARM_SMCCC_STD_CALL 0 -#define ARM_SMCCC_FAST_CALL 1 +#define ARM_SMCCC_STD_CALL _AC(0,U) +#define ARM_SMCCC_FAST_CALL _AC(1,U) #define ARM_SMCCC_TYPE_SHIFT 31 #define ARM_SMCCC_SMC_32 0 From patchwork Mon Feb 26 08:20:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129604 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3361206lja; Mon, 26 Feb 2018 00:29:57 -0800 (PST) X-Google-Smtp-Source: AH8x224byQuUymbhyU7eZA6c4FF8CECNikeWEaFa2QLZjNToTDuZmzEG0uTQeVhsapQNi4cQqgLP X-Received: by 10.99.95.81 with SMTP id t78mr7979990pgb.380.1519633797582; Mon, 26 Feb 2018 00:29:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633797; cv=none; d=google.com; s=arc-20160816; b=MjDmcJaVsH60ZlrWPaIfH8WbH41uTydfk9jlFqmBjTt7WotK8wAmFz4hs4iiWFouGq BUGKQlcFBFlChDjTpBNH8qBaBdfs1brVxKBYrI5/N1VfCEx47Oep7yobdbygDISPX/NY 4Ge5u87BMjyxa6feLi/KusFLjV/jIW1pguvp9LIkOSBANmu2JVkTVrvuiRgwH4ZnqLGl l6s1ervhiUNJau3UrcMePka8nlEjFWeLbX6L7yBjl6Wo4Ve3ubQTpPLj3OxR0Nv7EHKw po0QlMlT7ucM/k51u7G6UWWZYNPKPitWPW0ftq/pqTUArGgiB1rzNw0htu9LiTs+W0T2 X0iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=Jk3msMDfhwzjnG4JcoVxRkkc9wQeB0h3mvpSGrDHyMI=; b=QMvxOf8S46yjMlAxDE0nFBCk5OK7DHA8+CppGD+SPy2k2Nnb451cwhHvfpri3ei90Q vA/4Zvk1jWyeKWuVZJg0EqEJTW10K3t0ab3Uyqtx6z9g035inYQyIubD+RfO1dy4Xf3C unXQFajOg1sAWO03IN+6RaYcQ3Js/FWsd8cwIFVKg9JRV/zeau17YJ8Mh7sZ/pgvChOH xfXuYokk1AGzNTFbn+d2SghCr06sr0qh1AnNwqb5mbcgJebSB+Eb3IfgYVDuHVucVRCF jrNRgGhtjQPpfsDt5xQey3weqKSAB6Ih4BrkodpoShmBH4ALXXrT3D27Ts5vrAELaOGn uvLA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K2VXzaPB; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v1-v6si4437580ply.788.2018.02.26.00.29.57; Mon, 26 Feb 2018 00:29:57 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=K2VXzaPB; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752769AbeBZI0t (ORCPT + 28 others); Mon, 26 Feb 2018 03:26:49 -0500 Received: from mail-pg0-f67.google.com ([74.125.83.67]:38718 "EHLO mail-pg0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751926AbeBZI0n (ORCPT ); Mon, 26 Feb 2018 03:26:43 -0500 Received: by mail-pg0-f67.google.com with SMTP id l24so5917424pgc.5 for ; Mon, 26 Feb 2018 00:26:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=Jk3msMDfhwzjnG4JcoVxRkkc9wQeB0h3mvpSGrDHyMI=; b=K2VXzaPBBGvN2xHx0iVSmetb/18MENzUuuS5prx1YuYsYrE9hgOmQA31MV61qpcNkn /MGcwubdwJi4K6BrTEA6uAgiVln2fnTOM268TYEMPxdnDD+UxCwclN3Q8RNV6xW96W+H hfoyY4R7Osz5QH2/VlcGi68fH6P618aNKznn0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=Jk3msMDfhwzjnG4JcoVxRkkc9wQeB0h3mvpSGrDHyMI=; b=iVVGviyV8j3Adk+6M3jXvrWKDtJvysp3KR2xufcbbGLcwRy4EYeOXNMO3ez+K98l6T uxFZBCHphB+XGc+AA58lRtjvy5r1FC+vkP4yYxFoYKzhFFp0adjp1hfPZkTpGBZd2S9Y EO5ae491rr+gMlwJ9EjlPZmi4nOIg/zJB7awQQt+pDx2J6w/ZxBEuMXbMzMA0dHX0hqK Lna/JuoJfogk1XMlD2n/q9A8aj0XJc04CuAeBl6WyxaMbhKJjB/WNq3AeQYBf5MvVaae Zybr+tgmF0/8YizdFJ1cO0wdardafOrNcS//u2ru1IETl0OC5c8f2jvZiN3AkSac5Za6 5CHA== X-Gm-Message-State: APf1xPDdz4pjtQxACDCRyKTESV8vvvrJFXvnX5RryVLN8kDy1XZ59yTM 98cRBIBJ/mAny/rSeIv5oC+iWLOesuc= X-Received: by 10.101.70.203 with SMTP id n11mr7544334pgr.377.1519633602876; Mon, 26 Feb 2018 00:26:42 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:42 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-kernel@vger.kernel.org (open list) Subject: [PATCH 43/52] arm/arm64: smccc: Implement SMCCC v1.1 inline primitive Date: Mon, 26 Feb 2018 16:20:17 +0800 Message-Id: <1519633227-29832-44-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit f2d3b2e8759a upstream. One of the major improvement of SMCCC v1.1 is that it only clobbers the first 4 registers, both on 32 and 64bit. This means that it becomes very easy to provide an inline version of the SMC call primitive, and avoid performing a function call to stash the registers that would otherwise be clobbered by SMCCC v1.0. Reviewed-by: Robin Murphy Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- include/linux/arm-smccc.h | 141 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 141 insertions(+) -- 2.7.4 diff --git a/include/linux/arm-smccc.h b/include/linux/arm-smccc.h index dd44d84..a031897 100644 --- a/include/linux/arm-smccc.h +++ b/include/linux/arm-smccc.h @@ -150,5 +150,146 @@ asmlinkage void __arm_smccc_hvc(unsigned long a0, unsigned long a1, #define arm_smccc_hvc_quirk(...) __arm_smccc_hvc(__VA_ARGS__) +/* SMCCC v1.1 implementation madness follows */ +#ifdef CONFIG_ARM64 + +#define SMCCC_SMC_INST "smc #0" +#define SMCCC_HVC_INST "hvc #0" + +#elif defined(CONFIG_ARM) +#include +#include + +#define SMCCC_SMC_INST __SMC(0) +#define SMCCC_HVC_INST __HVC(0) + +#endif + +#define ___count_args(_0, _1, _2, _3, _4, _5, _6, _7, _8, x, ...) x + +#define __count_args(...) \ + ___count_args(__VA_ARGS__, 7, 6, 5, 4, 3, 2, 1, 0) + +#define __constraint_write_0 \ + "+r" (r0), "=&r" (r1), "=&r" (r2), "=&r" (r3) +#define __constraint_write_1 \ + "+r" (r0), "+r" (r1), "=&r" (r2), "=&r" (r3) +#define __constraint_write_2 \ + "+r" (r0), "+r" (r1), "+r" (r2), "=&r" (r3) +#define __constraint_write_3 \ + "+r" (r0), "+r" (r1), "+r" (r2), "+r" (r3) +#define __constraint_write_4 __constraint_write_3 +#define __constraint_write_5 __constraint_write_4 +#define __constraint_write_6 __constraint_write_5 +#define __constraint_write_7 __constraint_write_6 + +#define __constraint_read_0 +#define __constraint_read_1 +#define __constraint_read_2 +#define __constraint_read_3 +#define __constraint_read_4 "r" (r4) +#define __constraint_read_5 __constraint_read_4, "r" (r5) +#define __constraint_read_6 __constraint_read_5, "r" (r6) +#define __constraint_read_7 __constraint_read_6, "r" (r7) + +#define __declare_arg_0(a0, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register unsigned long r1 asm("r1"); \ + register unsigned long r2 asm("r2"); \ + register unsigned long r3 asm("r3") + +#define __declare_arg_1(a0, a1, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register typeof(a1) r1 asm("r1") = a1; \ + register unsigned long r2 asm("r2"); \ + register unsigned long r3 asm("r3") + +#define __declare_arg_2(a0, a1, a2, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register typeof(a1) r1 asm("r1") = a1; \ + register typeof(a2) r2 asm("r2") = a2; \ + register unsigned long r3 asm("r3") + +#define __declare_arg_3(a0, a1, a2, a3, res) \ + struct arm_smccc_res *___res = res; \ + register u32 r0 asm("r0") = a0; \ + register typeof(a1) r1 asm("r1") = a1; \ + register typeof(a2) r2 asm("r2") = a2; \ + register typeof(a3) r3 asm("r3") = a3 + +#define __declare_arg_4(a0, a1, a2, a3, a4, res) \ + __declare_arg_3(a0, a1, a2, a3, res); \ + register typeof(a4) r4 asm("r4") = a4 + +#define __declare_arg_5(a0, a1, a2, a3, a4, a5, res) \ + __declare_arg_4(a0, a1, a2, a3, a4, res); \ + register typeof(a5) r5 asm("r5") = a5 + +#define __declare_arg_6(a0, a1, a2, a3, a4, a5, a6, res) \ + __declare_arg_5(a0, a1, a2, a3, a4, a5, res); \ + register typeof(a6) r6 asm("r6") = a6 + +#define __declare_arg_7(a0, a1, a2, a3, a4, a5, a6, a7, res) \ + __declare_arg_6(a0, a1, a2, a3, a4, a5, a6, res); \ + register typeof(a7) r7 asm("r7") = a7 + +#define ___declare_args(count, ...) __declare_arg_ ## count(__VA_ARGS__) +#define __declare_args(count, ...) ___declare_args(count, __VA_ARGS__) + +#define ___constraints(count) \ + : __constraint_write_ ## count \ + : __constraint_read_ ## count \ + : "memory" +#define __constraints(count) ___constraints(count) + +/* + * We have an output list that is not necessarily used, and GCC feels + * entitled to optimise the whole sequence away. "volatile" is what + * makes it stick. + */ +#define __arm_smccc_1_1(inst, ...) \ + do { \ + __declare_args(__count_args(__VA_ARGS__), __VA_ARGS__); \ + asm volatile(inst "\n" \ + __constraints(__count_args(__VA_ARGS__))); \ + if (___res) \ + *___res = (typeof(*___res)){r0, r1, r2, r3}; \ + } while (0) + +/* + * arm_smccc_1_1_smc() - make an SMCCC v1.1 compliant SMC call + * + * This is a variadic macro taking one to eight source arguments, and + * an optional return structure. + * + * @a0-a7: arguments passed in registers 0 to 7 + * @res: result values from registers 0 to 3 + * + * This macro is used to make SMC calls following SMC Calling Convention v1.1. + * The content of the supplied param are copied to registers 0 to 7 prior + * to the SMC instruction. The return values are updated with the content + * from register 0 to 3 on return from the SMC instruction if not NULL. + */ +#define arm_smccc_1_1_smc(...) __arm_smccc_1_1(SMCCC_SMC_INST, __VA_ARGS__) + +/* + * arm_smccc_1_1_hvc() - make an SMCCC v1.1 compliant HVC call + * + * This is a variadic macro taking one to eight source arguments, and + * an optional return structure. + * + * @a0-a7: arguments passed in registers 0 to 7 + * @res: result values from registers 0 to 3 + * + * This macro is used to make HVC calls following SMC Calling Convention v1.1. + * The content of the supplied param are copied to registers 0 to 7 prior + * to the HVC instruction. The return values are updated with the content + * from register 0 to 3 on return from the HVC instruction if not NULL. + */ +#define arm_smccc_1_1_hvc(...) __arm_smccc_1_1(SMCCC_HVC_INST, __VA_ARGS__) + #endif /*__ASSEMBLY__*/ #endif /*__LINUX_ARM_SMCCC_H*/ From patchwork Mon Feb 26 08:20:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129595 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359314lja; Mon, 26 Feb 2018 00:27:02 -0800 (PST) X-Google-Smtp-Source: AH8x226DZjzMnV7Cd4/gesDzCkF89V7ibAd/u2okn58gLgxeEI4cwx5HrzkRGrgVv6xYpnpqAkN9 X-Received: by 2002:a17:902:8d81:: with SMTP id v1-v6mr9823761plo.430.1519633622687; Mon, 26 Feb 2018 00:27:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633622; cv=none; d=google.com; s=arc-20160816; b=kv+MVCKUEeRtfIhmhL2eL2VJJppaijntlDfjh8MeLQhpkgyGaUhsEUJoeqkh48SzBd 2qI9iPzlc8oyIE++nbDevH5HQQo2yZoxhat1sa4ezChUWuEwuIkW65MrtJhbmXlpLYRr a8iHJ4GCWa3VwQHTB2GKqJZpaXk64m0dTm20jABuT7rco4NX6tfO5sHZERRJwue+RL+n DlIZKqp2h6PP/p9GXZqqa58SkOwca/HZqli6sl8m2Y83RLwt841atfyWnqHa1xVkc+4+ x5tc/pBPQPYt4ZIj2wSgEcS5VDyvJWLbGSE2dG12QbK9pYSRuYyyTqHY5nWxW/POtE19 AVFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=IG03JAN328WQoLPMHLwAVvntjAGN6WojvlYcD547iug=; b=LxFjqvGW0gO0W5AbX9s+xYsPFWRD86CQou5awEKgGeIZ36xE+Pl6W23sVqlIvM90VX 3YbtyzOKUBDGOV/VB0KAW0wHPmSSPOmDZdwHR9yo+SR99gzZARrktH6Fe/T6aNX5pm1o XIjRbk7GcdbT9dzyVDU9TFdh+JMKGNMg8LwJm5K63V1AvbJ8GlDDiYU7xFwnX3y7HJWM FVtAT43ITgr4GsfrE9bTUYk/x0YIwJyDknFFuqu41nEkr4NkUA2LRzIjfZB/olCcQ9gi N61wgzMtu8snTe3RJQEc2v8yxHxxtSsVlX6UTqeNpscaFQuN2LsqeS0kDgyDdw/1visj BGnA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ivfKy5AL; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a22-v6si5233862plm.773.2018.02.26.00.27.02; Mon, 26 Feb 2018 00:27:02 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ivfKy5AL; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752801AbeBZI07 (ORCPT + 28 others); Mon, 26 Feb 2018 03:26:59 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:36131 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752773AbeBZI0v (ORCPT ); Mon, 26 Feb 2018 03:26:51 -0500 Received: by mail-pf0-f194.google.com with SMTP id 68so6230508pfx.3 for ; Mon, 26 Feb 2018 00:26:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=IG03JAN328WQoLPMHLwAVvntjAGN6WojvlYcD547iug=; b=ivfKy5ALTV8W9jS9/aq+RYhNJoCWcBm9h5ZHNs+gFuoEXXjaK5sQOXmlZPe/KpcqX3 BGojPU5Zqnyyt6yEnZWwVnLRzA/2VSg9iW1rmRf0SO0QkG7VEllJZGB5EmJgniiU/ZxX BZo5h8HIxfhBikGXQhU7Y0r0VxrqsutZg5ZhI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=IG03JAN328WQoLPMHLwAVvntjAGN6WojvlYcD547iug=; b=iMW92mh488E3fE6qSOlOlT3HcH70Wjc61y5L4y8LcPZYNDz6Sk6cYpAlirlz8Y5sLA F/pMfXjkha4hJfj2TA42x+xHMXaKz78d+l7g1FKw9z0GDS8nuoJiqf6Jtad4tVCQXLNk Dj8UfvNH/mLtnINGw8cjIjKICp51k5RfdrjcghUvlMT2DohSoVc8JtRIerbLGIx2WVfH JE0h5/qv2ekKF87k81ZZGGBSTKAP4OjMWNL2vZ6qxFalJn9dQaXz8ar3DdvA7CEq4ww+ f8jgZuQADtqY6xcoRiQRWTvGqK9J/Oy2Ar/PnNzuPIvHMioYRzCXTSI8VKSe1pECkX0n HDIw== X-Gm-Message-State: APf1xPA3mTWQyRre7FRjvtUjZ4/sBPP77TI500xPaWL6e8Qw21BHu8sR DrgR/Ow3ykaWCC01aLWXIfXa5w== X-Received: by 10.99.0.207 with SMTP id 198mr6260504pga.364.1519633611030; Mon, 26 Feb 2018 00:26:51 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:26:50 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 44/52] arm64: Add ARM_SMCCC_ARCH_WORKAROUND_1 BP hardening support Date: Mon, 26 Feb 2018 16:20:18 +0800 Message-Id: <1519633227-29832-45-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit b092201e0020 upstream. Add the detection and runtime code for ARM_SMCCC_ARCH_WORKAROUND_1. It is lovely. Really. Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no qcom hyp functions in arch/arm64/kernel/bpi.S arch/arm64/kernel/cpu_errata.c --- arch/arm64/kernel/bpi.S | 20 ++++++++++++ arch/arm64/kernel/cpu_errata.c | 72 ++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 89 insertions(+), 3 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index dec95bd..c72f261 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -17,6 +17,7 @@ */ #include +#include .macro ventry target .rept 31 @@ -77,3 +78,22 @@ ENTRY(__psci_hyp_bp_inval_start) ldp x0, x1, [sp, #(16 * 8)] add sp, sp, #(8 * 18) ENTRY(__psci_hyp_bp_inval_end) + +.macro smccc_workaround_1 inst + sub sp, sp, #(8 * 4) + stp x2, x3, [sp, #(8 * 0)] + stp x0, x1, [sp, #(8 * 2)] + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 + \inst #0 + ldp x2, x3, [sp, #(8 * 0)] + ldp x0, x1, [sp, #(8 * 2)] + add sp, sp, #(8 * 4) +.endm + +ENTRY(__smccc_workaround_1_smc_start) + smccc_workaround_1 smc +ENTRY(__smccc_workaround_1_smc_end) + +ENTRY(__smccc_workaround_1_hvc_start) + smccc_workaround_1 hvc +ENTRY(__smccc_workaround_1_hvc_end) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index f8810bf..9632319 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -54,6 +54,10 @@ DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); #ifdef CONFIG_KVM extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; +extern char __smccc_workaround_1_smc_start[]; +extern char __smccc_workaround_1_smc_end[]; +extern char __smccc_workaround_1_hvc_start[]; +extern char __smccc_workaround_1_hvc_end[]; static void __copy_hyp_vect_bpi(int slot, const char *hyp_vecs_start, const char *hyp_vecs_end) @@ -96,8 +100,12 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn, spin_unlock(&bp_lock); } #else -#define __psci_hyp_bp_inval_start NULL -#define __psci_hyp_bp_inval_end NULL +#define __psci_hyp_bp_inval_start NULL +#define __psci_hyp_bp_inval_end NULL +#define __smccc_workaround_1_smc_start NULL +#define __smccc_workaround_1_smc_end NULL +#define __smccc_workaround_1_hvc_start NULL +#define __smccc_workaround_1_hvc_end NULL static void __install_bp_hardening_cb(bp_hardening_cb_t fn, const char *hyp_vecs_start, @@ -124,17 +132,75 @@ static void install_bp_hardening_cb(const struct arm64_cpu_capabilities *entry, __install_bp_hardening_cb(fn, hyp_vecs_start, hyp_vecs_end); } +#include +#include #include +static void call_smc_arch_workaround_1(void) +{ + arm_smccc_1_1_smc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); +} + +static void call_hvc_arch_workaround_1(void) +{ + arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); +} + +static bool check_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry) +{ + bp_hardening_cb_t cb; + void *smccc_start, *smccc_end; + struct arm_smccc_res res; + + if (!entry->matches(entry, SCOPE_LOCAL_CPU)) + return false; + + if (psci_ops.smccc_version == SMCCC_VERSION_1_0) + return false; + + switch (psci_ops.conduit) { + case PSCI_CONDUIT_HVC: + arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_ARCH_WORKAROUND_1, &res); + if (res.a0) + return false; + cb = call_hvc_arch_workaround_1; + smccc_start = __smccc_workaround_1_hvc_start; + smccc_end = __smccc_workaround_1_hvc_end; + break; + + case PSCI_CONDUIT_SMC: + arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, + ARM_SMCCC_ARCH_WORKAROUND_1, &res); + if (res.a0) + return false; + cb = call_smc_arch_workaround_1; + smccc_start = __smccc_workaround_1_smc_start; + smccc_end = __smccc_workaround_1_smc_end; + break; + + default: + return false; + } + + install_bp_hardening_cb(entry, cb, smccc_start, smccc_end); + + return true; +} + static int enable_psci_bp_hardening(void *data) { const struct arm64_cpu_capabilities *entry = data; - if (psci_ops.get_version) + if (psci_ops.get_version) { + if (check_smccc_arch_workaround_1(entry)) + return 0; + install_bp_hardening_cb(entry, (bp_hardening_cb_t)psci_ops.get_version, __psci_hyp_bp_inval_start, __psci_hyp_bp_inval_end); + } return 0; } From patchwork Mon Feb 26 08:20:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129596 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359465lja; Mon, 26 Feb 2018 00:27:17 -0800 (PST) X-Google-Smtp-Source: AH8x227NKza38lIoxLamJP2kXSxeVlXXz6EN/SNNbYiF93S9UBHGnvCk7QdsipHEDcgyMrHzRiYR X-Received: by 2002:a17:902:8f97:: with SMTP id z23-v6mr9928329plo.162.1519633637755; Mon, 26 Feb 2018 00:27:17 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633637; cv=none; d=google.com; s=arc-20160816; b=VkFFnKC6OG+8Nt1T38ahI7pk16PKFE60OhOsJVn6gtYSTGmX4kleQgG1igGQ6a81zB VSrTJs9c7FKa6EYzi/0lYjdMvZNWQC0hiEOd8C0WAK2Xtw/nMXuQVgeYYBe17kjHZ8Cu Fkpd5Q84FSqTgPkw0Lu4vk9o2IAsE2jbE5kb9KlvHVjl3nIwgC5s9MdnjEJxYUdx7bxP 3J/MsJ9ZVu5Ki4+W5stXbgf6V+A6rLW0hY2e8fvkBeLgTfABrdE8L9U9JJedFU4f4dBX xOBVroz6mqyKFE1gKa7HlqMybjy+xWuOzJSx8fQAJdVPnbl4oBR30Dep+RzleIInhf6o cI3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=FrYQc7jNWo3H77WRAN83meLx1vfqAFkomM5c8aD6cAQ=; b=APAy9XqCKqq/RJ0BYfl+OTQ7p4mEnuleEsqT0zycFm3uxqRjHsFdEE6yzaMfcndy/+ 4hk5Tya1B+uu1O/Ag3wPHakl9sGp+hEpkPSfWyb6AUzDYCq52ZRDs9UqPsDNW9wJeh/m 1cZjwHJ+ktVeAze3re50xtEHju1BIZy/rtjupfpunOq3EnxS94SDa6yCWyzXSlmZOAje LadxvObPr1FZ5bpqRVnPGqQmtAGLmHP0O1YGqunOR8kCQlIzMB0COufspheiCATVDVkf y5xkMlxKu8ZskZCzsg/JuBHKyGX7hTO+jdeYKLiPuuO6xBfFz0Z4qJkKbkPBGK2L03Lw IIzA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PMncHEnw; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id s190si5281566pgc.510.2018.02.26.00.27.17; Mon, 26 Feb 2018 00:27:17 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PMncHEnw; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752840AbeBZI1O (ORCPT + 28 others); Mon, 26 Feb 2018 03:27:14 -0500 Received: from mail-pf0-f196.google.com ([209.85.192.196]:40189 "EHLO mail-pf0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752804AbeBZI1B (ORCPT ); Mon, 26 Feb 2018 03:27:01 -0500 Received: by mail-pf0-f196.google.com with SMTP id m5so6219285pff.7 for ; Mon, 26 Feb 2018 00:27:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=FrYQc7jNWo3H77WRAN83meLx1vfqAFkomM5c8aD6cAQ=; b=PMncHEnwzW+oNl0sJH7ZLS5EK4JoEBtJGeg9w9/zhui1CandURqm0suWxeiXDh7FLc lKIIJsDxvJqVKtIGyda3nQd5HYUOJ6XeKO0cbNDuJrXp9QauHuFpj7/JoNetQGK3dilp lvcIAy+4IJJsnKUAWBuhoygJ+eMujemLUAjNg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=FrYQc7jNWo3H77WRAN83meLx1vfqAFkomM5c8aD6cAQ=; b=OII0QXXSmfkVZMAoBuFgDDpeoGewQVwm8kH+mCYX8gCncGy56EJiWNFKVLMDBUcuKy FwWEPiLiNO7nGAYFwLaRZOKwbufiI6XJ6UGOO7xPvrkCTETfZhEn9xQqvdzjhrgHKoAd /op+9wNg0Isv5kTpxc/JI5eczwa4728NXlBzx/m5rn5qL4IwP2gY5gAUFoLNCAKyGYDK HpbMj5+2KAvfVMDu6aN/0b0qqyqyuFzvleaCIi+vMdT8BbGy0I16wb5JZyIsHu1u+jQs ei+ZeZPVNzqAzo6PvdAnNeqrhvHauo+ezZDrM2cmbJI7qPY8fLQP98+NSZ2bXydP5Udq vRXA== X-Gm-Message-State: APf1xPB1C4YAoLvr0eLza5pC9cd/rRWsBGmy4GmyEnhNqGttCavV5afF 19n+2A4V5n2/Y5aVTadiTA5fQg== X-Received: by 10.98.28.202 with SMTP id c193mr9816366pfc.109.1519633620617; Mon, 26 Feb 2018 00:27:00 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.26.51 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:00 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Christoffer Dall , Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , linux-arm-kernel@lists.infradead.org (moderated list:ARM64 PORT (AARCH64 ARCHITECTURE)), linux-kernel@vger.kernel.org (open list), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)), kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)) Subject: [PATCH 45/52] arm64: Kill PSCI_GET_VERSION as a variant-2 workaround Date: Mon, 26 Feb 2018 16:20:19 +0800 Message-Id: <1519633227-29832-46-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier commit 3a0a397ff5ff upstream. Now that we've standardised on SMCCC v1.1 to perform the branch prediction invalidation, let's drop the previous band-aid. If vendors haven't updated their firmware to do SMCCC 1.1, they haven't updated PSCI either, so we don't loose anything. Tested-by: Ard Biesheuvel Signed-off-by: Marc Zyngier Signed-off-by: Catalin Marinas Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no falkor/thunderx2/vulcan in arch/arm64/kernel/cpu_errata.c --- arch/arm64/kernel/bpi.S | 24 ------------------------ arch/arm64/kernel/cpu_errata.c | 41 +++++++++++------------------------------ arch/arm64/kvm/hyp/switch.c | 14 -------------- 3 files changed, 11 insertions(+), 68 deletions(-) -- 2.7.4 diff --git a/arch/arm64/kernel/bpi.S b/arch/arm64/kernel/bpi.S index c72f261..dc4eb15 100644 --- a/arch/arm64/kernel/bpi.S +++ b/arch/arm64/kernel/bpi.S @@ -54,30 +54,6 @@ ENTRY(__bp_harden_hyp_vecs_start) vectors __kvm_hyp_vector .endr ENTRY(__bp_harden_hyp_vecs_end) -ENTRY(__psci_hyp_bp_inval_start) - sub sp, sp, #(8 * 18) - stp x16, x17, [sp, #(16 * 0)] - stp x14, x15, [sp, #(16 * 1)] - stp x12, x13, [sp, #(16 * 2)] - stp x10, x11, [sp, #(16 * 3)] - stp x8, x9, [sp, #(16 * 4)] - stp x6, x7, [sp, #(16 * 5)] - stp x4, x5, [sp, #(16 * 6)] - stp x2, x3, [sp, #(16 * 7)] - stp x0, x1, [sp, #(16 * 8)] - mov x0, #0x84000000 - smc #0 - ldp x16, x17, [sp, #(16 * 0)] - ldp x14, x15, [sp, #(16 * 1)] - ldp x12, x13, [sp, #(16 * 2)] - ldp x10, x11, [sp, #(16 * 3)] - ldp x8, x9, [sp, #(16 * 4)] - ldp x6, x7, [sp, #(16 * 5)] - ldp x4, x5, [sp, #(16 * 6)] - ldp x2, x3, [sp, #(16 * 7)] - ldp x0, x1, [sp, #(16 * 8)] - add sp, sp, #(8 * 18) -ENTRY(__psci_hyp_bp_inval_end) .macro smccc_workaround_1 inst sub sp, sp, #(8 * 4) diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 9632319..8b74f80 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -53,7 +53,6 @@ static int cpu_enable_trap_ctr_access(void *__unused) DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data); #ifdef CONFIG_KVM -extern char __psci_hyp_bp_inval_start[], __psci_hyp_bp_inval_end[]; extern char __smccc_workaround_1_smc_start[]; extern char __smccc_workaround_1_smc_end[]; extern char __smccc_workaround_1_hvc_start[]; @@ -100,8 +99,6 @@ static void __install_bp_hardening_cb(bp_hardening_cb_t fn, spin_unlock(&bp_lock); } #else -#define __psci_hyp_bp_inval_start NULL -#define __psci_hyp_bp_inval_end NULL #define __smccc_workaround_1_smc_start NULL #define __smccc_workaround_1_smc_end NULL #define __smccc_workaround_1_hvc_start NULL @@ -146,24 +143,25 @@ static void call_hvc_arch_workaround_1(void) arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_WORKAROUND_1, NULL); } -static bool check_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *entry) +static int enable_smccc_arch_workaround_1(void *data) { + const struct arm64_cpu_capabilities *entry = data; bp_hardening_cb_t cb; void *smccc_start, *smccc_end; struct arm_smccc_res res; if (!entry->matches(entry, SCOPE_LOCAL_CPU)) - return false; + return 0; if (psci_ops.smccc_version == SMCCC_VERSION_1_0) - return false; + return 0; switch (psci_ops.conduit) { case PSCI_CONDUIT_HVC: arm_smccc_1_1_hvc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, ARM_SMCCC_ARCH_WORKAROUND_1, &res); if (res.a0) - return false; + return 0; cb = call_hvc_arch_workaround_1; smccc_start = __smccc_workaround_1_hvc_start; smccc_end = __smccc_workaround_1_hvc_end; @@ -173,35 +171,18 @@ static bool check_smccc_arch_workaround_1(const struct arm64_cpu_capabilities *e arm_smccc_1_1_smc(ARM_SMCCC_ARCH_FEATURES_FUNC_ID, ARM_SMCCC_ARCH_WORKAROUND_1, &res); if (res.a0) - return false; + return 0; cb = call_smc_arch_workaround_1; smccc_start = __smccc_workaround_1_smc_start; smccc_end = __smccc_workaround_1_smc_end; break; default: - return false; + return 0; } install_bp_hardening_cb(entry, cb, smccc_start, smccc_end); - return true; -} - -static int enable_psci_bp_hardening(void *data) -{ - const struct arm64_cpu_capabilities *entry = data; - - if (psci_ops.get_version) { - if (check_smccc_arch_workaround_1(entry)) - return 0; - - install_bp_hardening_cb(entry, - (bp_hardening_cb_t)psci_ops.get_version, - __psci_hyp_bp_inval_start, - __psci_hyp_bp_inval_end); - } - return 0; } #endif /* CONFIG_HARDEN_BRANCH_PREDICTOR */ @@ -301,22 +282,22 @@ const struct arm64_cpu_capabilities arm64_errata[] = { { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A73), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, { .capability = ARM64_HARDEN_BRANCH_PREDICTOR, MIDR_ALL_VERSIONS(MIDR_CORTEX_A75), - .enable = enable_psci_bp_hardening, + .enable = enable_smccc_arch_workaround_1, }, #endif { diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 996328e..154b471 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -311,20 +311,6 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) if (exit_code == ARM_EXCEPTION_TRAP && !__populate_fault_info(vcpu)) goto again; - if (exit_code == ARM_EXCEPTION_TRAP && - (kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC64 || - kvm_vcpu_trap_get_class(vcpu) == ESR_ELx_EC_HVC32)) { - u32 val = vcpu_get_reg(vcpu, 0); - - if (val == PSCI_0_2_FN_PSCI_VERSION) { - val = kvm_psci_version(vcpu, kern_hyp_va(vcpu->kvm)); - if (unlikely(val == KVM_ARM_PSCI_0_1)) - val = PSCI_RET_NOT_SUPPORTED; - vcpu_set_reg(vcpu, 0, val); - goto again; - } - } - if (static_branch_unlikely(&vgic_v2_cpuif_trap) && exit_code == ARM_EXCEPTION_TRAP) { bool valid; From patchwork Mon Feb 26 08:20:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129603 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3360830lja; Mon, 26 Feb 2018 00:29:20 -0800 (PST) X-Google-Smtp-Source: AH8x226GvvBtadCIRCZ5yXAEqECaEC1gB410GU2iAEBTx6mF/KCKdhjWnAR8ak3r5tTFrGIuMDM6 X-Received: by 2002:a17:902:1:: with SMTP id 1-v6mr9897218pla.187.1519633760416; Mon, 26 Feb 2018 00:29:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633760; cv=none; d=google.com; s=arc-20160816; b=zBnPgUZq9QiktMqStjvYNhvBlZEDr0DCpvkJmrUcyuT9Y8u0OA9p1UImXOt+EFg3Qg eZfvuZamQJhuzhYRl6DnpNyBJuKdQVOK4tKObD2T7+m3c+CVXt69rIUdy2iB8IRBGfwr U6ASBFksXvhF1FvRB0/d+i62rIAV5XGAWzOJEfCdjTbUNdhEZaPoJNjKC3yYk+dFK530 XaOeH108eHer8Loq10IKtvv9NcqqHmfTmRgoXg0hCJmO/oHlBH/iFGEL1bSaM07vAoMV 3r89XP3gokEjcjwcEDbJWngojpKIFKifzh3at4a/vwNFUseY2WDFD2xMR4Gt56yDo72z JjIQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=GbyVFSuKqlJlOZQir9uMnmIFVEqzFtVXTBGR5PiLN7c=; b=CG0r5KB6g+9bh0I0HEmUDRNUq0v2z9LcewTYnH+UKeXYe4sv+SaeUlalK9HWl72AIc /a9lorYFIKaYBwmaebmCtlrlar4m/ZthnlriMrwGh5UgFoBlbAS+7lHWoQlwiNwYlxue HZx856cfDK0VcsGejz4Vwv/zjgWLfOgkeKJLYBNtPJ0/UkA6BnkwKffqJ7cLSNGgthQK H18vnil+Cb6KEES2unUwrK3UkaEW/Dn4P4AqGkbkMSxIsG0LUSlYuDg2o4FPQ63m9kNq YqpiKZ2W/SsMABlMf8mw3Y6TQuvl+jDXOgnYxISsI8scrMXjHhJhljAouGxnDzFiPMEl zokA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Ghh1zawi; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a6si5826251pfi.123.2018.02.26.00.29.20; Mon, 26 Feb 2018 00:29:20 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Ghh1zawi; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752816AbeBZI3Q (ORCPT + 28 others); Mon, 26 Feb 2018 03:29:16 -0500 Received: from mail-pf0-f194.google.com ([209.85.192.194]:45679 "EHLO mail-pf0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752817AbeBZI1I (ORCPT ); Mon, 26 Feb 2018 03:27:08 -0500 Received: by mail-pf0-f194.google.com with SMTP id j24so6219691pff.12 for ; Mon, 26 Feb 2018 00:27:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=GbyVFSuKqlJlOZQir9uMnmIFVEqzFtVXTBGR5PiLN7c=; b=Ghh1zawid0gSoCGWFqZ8Xb6rZqmAU5LQ459FP88Qvqk4DvZetI4SEvpNNOqJQbrKpm i6K+u9tFTsR9ILLle8Zs2mGQa32YSDGTIyp9UycYqpsOSZofW75CmwPsoXZpPw4115+Q fHYNP8PZVKr03gNviVa3clg9PtWhcx/HDiIgY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=GbyVFSuKqlJlOZQir9uMnmIFVEqzFtVXTBGR5PiLN7c=; b=JMXg38bDexsgSsyPl0fz6b7ve0Do+xHlgrOZErwHaja72KRXQHvj/Mr8t2JaQQnWr+ MH07Ra7r94ByqikJG2PUbZzt5uD0nEf1x1ggbpr8ofXnj5fGQnuM9JBDP/Cxw3iH4bd6 1KG6MkJVhtvS7wpPnDnhnkBqXAfgccLFE9mNSrChipwqntKL9T6bXncAEpvPjQ4K3/+G gMBFSBTyTNj/XzEIX0uSLCgMDBs564V+XhsQlOc3PKVPKib+mSx2LlvoGYBCpbWotOPB 47VaBryPr8bszpi5XNQwUujvpKs75HpwTwqahmWN2pkbo0QKwJHp+yCqh6HGMA5OTXcz pU7Q== X-Gm-Message-State: APf1xPAu/Kyz8jrokjjJPWfPa87Iy5xUcilyteC0NZapNUE+uhKr7ZCa wmFQeEywomJ0/SPc2+IU3X3ezg== X-Received: by 10.98.152.205 with SMTP id d74mr9878127pfk.115.1519633627393; Mon, 26 Feb 2018 00:27:07 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:06 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Russell King , linux-arm-kernel@lists.infradead.org (moderated list:ARM PORT), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 46/52] arm: Add BTB invalidation on switch_mm for Cortex-A9, A12 and A17 Date: Mon, 26 Feb 2018 16:20:20 +0800 Message-Id: <1519633227-29832-47-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to avoid aliasing attacks against the branch predictor, some implementations require to invalidate the BTB when switching from one user context to another. For this, we reuse the existing implementation for Cortex-A8, and apply it to A9, A12 and A17. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi --- arch/arm/mm/Kconfig | 17 +++++++++++++++++ arch/arm/mm/proc-v7-2level.S | 4 ++-- arch/arm/mm/proc-v7-3level.S | 5 +++++ arch/arm/mm/proc-v7.S | 30 ++++++++++++++++++++++++++++-- 4 files changed, 52 insertions(+), 4 deletions(-) -- 2.7.4 diff --git a/arch/arm/mm/Kconfig b/arch/arm/mm/Kconfig index c1799dd..1a5acee 100644 --- a/arch/arm/mm/Kconfig +++ b/arch/arm/mm/Kconfig @@ -1068,3 +1068,20 @@ config DEBUG_ALIGN_RODATA additional section-aligned split of rodata from kernel text so it can be made explicitly non-executable. This padding may waste memory space to gain the additional protection. + +config HARDEN_BRANCH_PREDICTOR + bool "Harden the branch predictor against aliasing attacks" if EXPERT + default y + help + Speculation attacks against some high-performance processors rely on + being able to manipulate the branch predictor for a victim context by + executing aliasing branches in the attacker context. Such attacks + can be partially mitigated against by clearing internal branch + predictor state and limiting the prediction logic in some situations. + + This config option will take CPU-specific actions to harden the + branch predictor against aliasing attacks and may rely on specific + instruction sequences or control bits being set by the system + firmware. + + If unsure, say Y. diff --git a/arch/arm/mm/proc-v7-2level.S b/arch/arm/mm/proc-v7-2level.S index c6141a5..0422e58b 100644 --- a/arch/arm/mm/proc-v7-2level.S +++ b/arch/arm/mm/proc-v7-2level.S @@ -41,7 +41,7 @@ * even on Cortex-A8 revisions not affected by 430973. * If IBE is not set, the flush BTAC/BTB won't do anything. */ -ENTRY(cpu_ca8_switch_mm) +ENTRY(cpu_v7_btbinv_switch_mm) #ifdef CONFIG_MMU mov r2, #0 mcr p15, 0, r2, c7, c5, 6 @ flush BTAC/BTB @@ -66,7 +66,7 @@ ENTRY(cpu_v7_switch_mm) #endif bx lr ENDPROC(cpu_v7_switch_mm) -ENDPROC(cpu_ca8_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) /* * cpu_v7_set_pte_ext(ptep, pte) diff --git a/arch/arm/mm/proc-v7-3level.S b/arch/arm/mm/proc-v7-3level.S index 5e5720e..a25450b 100644 --- a/arch/arm/mm/proc-v7-3level.S +++ b/arch/arm/mm/proc-v7-3level.S @@ -54,6 +54,10 @@ * Set the translation table base pointer to be pgd_phys (physical address of * the new TTB). */ +ENTRY(cpu_v7_btbinv_switch_mm) +#ifdef CONFIG_MMU + mcr p15, 0, r0, c7, c5, 6 @ flush BTAC/BTB +#endif ENTRY(cpu_v7_switch_mm) #ifdef CONFIG_MMU mmid r2, r2 @@ -64,6 +68,7 @@ ENTRY(cpu_v7_switch_mm) #endif ret lr ENDPROC(cpu_v7_switch_mm) +ENDPROC(cpu_v7_btbinv_switch_mm) #ifdef __ARMEB__ #define rl r3 diff --git a/arch/arm/mm/proc-v7.S b/arch/arm/mm/proc-v7.S index d00d52c..ff7018a 100644 --- a/arch/arm/mm/proc-v7.S +++ b/arch/arm/mm/proc-v7.S @@ -163,6 +163,7 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_ca8_dcache_clean_area, cpu_v7_dcache_clean_area globl_equ cpu_ca8_set_pte_ext, cpu_v7_set_pte_ext globl_equ cpu_ca8_suspend_size, cpu_v7_suspend_size + globl_equ cpu_ca8_switch_mm, cpu_v7_btbinv_switch_mm #ifdef CONFIG_ARM_CPU_SUSPEND globl_equ cpu_ca8_do_suspend, cpu_v7_do_suspend globl_equ cpu_ca8_do_resume, cpu_v7_do_resume @@ -176,7 +177,11 @@ ENDPROC(cpu_v7_do_resume) globl_equ cpu_ca9mp_reset, cpu_v7_reset globl_equ cpu_ca9mp_do_idle, cpu_v7_do_idle globl_equ cpu_ca9mp_dcache_clean_area, cpu_v7_dcache_clean_area +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + globl_equ cpu_ca9mp_switch_mm, cpu_v7_btbinv_switch_mm +#else globl_equ cpu_ca9mp_switch_mm, cpu_v7_switch_mm +#endif globl_equ cpu_ca9mp_set_pte_ext, cpu_v7_set_pte_ext .globl cpu_ca9mp_suspend_size .equ cpu_ca9mp_suspend_size, cpu_v7_suspend_size + 4 * 2 @@ -202,6 +207,26 @@ ENTRY(cpu_ca9mp_do_resume) ENDPROC(cpu_ca9mp_do_resume) #endif +/* + * Cortex-A12/A17 + */ + globl_equ cpu_ca17_proc_init, cpu_v7_proc_init + globl_equ cpu_ca17_proc_fin, cpu_v7_proc_fin + globl_equ cpu_ca17_reset, cpu_v7_reset + globl_equ cpu_ca17_do_idle, cpu_v7_do_idle + globl_equ cpu_ca17_dcache_clean_area, cpu_v7_dcache_clean_area + globl_equ cpu_ca17_set_pte_ext, cpu_v7_set_pte_ext + globl_equ cpu_ca17_suspend_size, cpu_v7_suspend_size +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + globl_equ cpu_ca17_switch_mm, cpu_v7_btbinv_switch_mm +#else + globl_equ cpu_ca17_switch_mm, cpu_v7_switch_mm +#endif +#ifdef CONFIG_ARM_CPU_SUSPEND + globl_equ cpu_ca17_do_suspend, cpu_v7_do_suspend + globl_equ cpu_ca17_do_resume, cpu_v7_do_resume +#endif + #ifdef CONFIG_CPU_PJ4B globl_equ cpu_pj4b_switch_mm, cpu_v7_switch_mm globl_equ cpu_pj4b_set_pte_ext, cpu_v7_set_pte_ext @@ -543,6 +568,7 @@ __v7_setup_stack: @ define struct processor (see and proc-macros.S) define_processor_functions v7, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 + define_processor_functions ca17, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 #ifndef CONFIG_ARM_LPAE define_processor_functions ca8, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 define_processor_functions ca9mp, dabort=v7_early_abort, pabort=v7_pabort, suspend=1 @@ -653,7 +679,7 @@ __v7_ca7mp_proc_info: __v7_ca12mp_proc_info: .long 0x410fc0d0 .long 0xff0ffff0 - __v7_proc __v7_ca12mp_proc_info, __v7_ca12mp_setup + __v7_proc __v7_ca12mp_proc_info, __v7_ca12mp_setup, proc_fns = ca17_processor_functions .size __v7_ca12mp_proc_info, . - __v7_ca12mp_proc_info /* @@ -683,7 +709,7 @@ __v7_b15mp_proc_info: __v7_ca17mp_proc_info: .long 0x410fc0e0 .long 0xff0ffff0 - __v7_proc __v7_ca17mp_proc_info, __v7_ca17mp_setup + __v7_proc __v7_ca17mp_proc_info, __v7_ca17mp_setup, proc_fns = ca17_processor_functions .size __v7_ca17mp_proc_info, . - __v7_ca17mp_proc_info /* From patchwork Mon Feb 26 08:20:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Shi X-Patchwork-Id: 129599 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp3359675lja; Mon, 26 Feb 2018 00:27:40 -0800 (PST) X-Google-Smtp-Source: AH8x226yLFsSfo/7JOCV8WHQXOUoQAgwgu8sbvdjjTa2VdHvuBai7Kb/Cc6qFzxRGQQnIikQRJs1 X-Received: by 10.98.31.155 with SMTP id l27mr9903668pfj.176.1519633659876; Mon, 26 Feb 2018 00:27:39 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519633659; cv=none; d=google.com; s=arc-20160816; b=AGQ1pDExRlI7L9GUVAa4A1oi9DssV5bqjgZaG5Su255TpNf+bxcTNELZKZfmRl4ZES NEh07M4SOiczUXhC+F5DhbgI1650A7baPn2z0AGh6Y0x/PN19QRv7lAhiXDOuDVmgSCt k11BDzY5XjP1a/gnh29W6xdeCMJ2MvFQTigWqGanhSaxVry8pXA6ohsWqW4U0VFOy87y vkVInmdAcmQYdm8ymIuHx2qL4NGaPrmbn09bPXdxGDIHCozf4hKISKE35IW7gzJe5Rdd iIHpMtfw/1uCq1JDWf2DtUwuOxvxH231Sszd7/eseZuT37ngFvNhaYqPoohIp0Jp0ASz U6lw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:to:from:dkim-signature:arc-authentication-results; bh=c+m8Xx/gdzZk3oAgyKLtXnF31YiTGhZDX4r8l10YuK4=; b=XbeDvzvfaikYC94Jne9dNa+ITDQslSliwgaSB7fdL2hplKMlkEValONwMejFmnzsVF 7Zw3BR5d+EI8HjxWJTUbgPgpzDvTSm1ee/ZwhhY6sXzvz2aj0HFuxyHEx3AECA6q8Jz4 KaDEIdsDGMSmhSsxu51nDgsUNpV+ZjLjMSkp+qFHKS4eoKQxUV9aGIn1BZSvJqfNI11w TMLI3VkXp40Vv6CcrmL30aXQ6e+WdzHOMetg8+l4PWzUSeuW4PWgE6vAGPmTtsSxP+TZ SnYpFcUcm/zU+tHPV5b6jt9YbdbXmmlR6MZkj03t0A6+ENBKN5nrZvyUVwqWjSjMzlyV u6kA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Zh8hg4a5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f23-v6si6600728plk.810.2018.02.26.00.27.39; Mon, 26 Feb 2018 00:27:39 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Zh8hg4a5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752472AbeBZI1f (ORCPT + 28 others); Mon, 26 Feb 2018 03:27:35 -0500 Received: from mail-pf0-f195.google.com ([209.85.192.195]:40206 "EHLO mail-pf0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752855AbeBZI10 (ORCPT ); Mon, 26 Feb 2018 03:27:26 -0500 Received: by mail-pf0-f195.google.com with SMTP id m5so6219574pff.7 for ; Mon, 26 Feb 2018 00:27:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:subject:date:message-id:in-reply-to:references; bh=c+m8Xx/gdzZk3oAgyKLtXnF31YiTGhZDX4r8l10YuK4=; b=Zh8hg4a58XLRH9ZUZZAbbTsxHhtmh4//H1Y1YjpNB5gPV7cEHDnDNISEwBOdXrdb++ yMUjG/wRHSN96UZxm3t2TswSbmQKyc3A5m/TgPbAvf8Kw9ovue+yVKjFR9UWJGwsPI/B 8mlmlRb1dvoiUnyus68owrxZSqzPkk+61tpa0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references; bh=c+m8Xx/gdzZk3oAgyKLtXnF31YiTGhZDX4r8l10YuK4=; b=ssvslbXXu334l8XxojyualQboEXN+CvzrTMB5Ja7cU6Q6Rg7lh7qosmOC3TDrdKJZe wO6ekHpmH3wn1w8MrdFDR/LIwXIABIFFx7VUVtY+JRdxdwyPiGCDzdaE4v4MQfDUxYXp hxmsqSiaO1r2zUos4Hw3igrkNZLiAv+4YSLlY25cW8WcKWw5iQY3Jyvm1UBBjMUq/SUi DxeZqOfSQ0xru/lF+tIW0V2W+tkdOwyKu9lVDTm8lHA1gxD66x8F3wn/n9GzRabN6FZ+ MyQrWWXT2HO3Ot82geCMvtERw+d15SWQkkF8XETAuLtHZ0hcSTOeSgO27Z/PCSALu2/b mwaQ== X-Gm-Message-State: APf1xPASS6vQxwGr52FE+Qb8bR90DphYSiePH8npHM6nwqZkVcI+kiCz cIRaTJs8JkUR+D4/60IFLct9ag== X-Received: by 10.98.227.10 with SMTP id g10mr9837192pfh.200.1519633645577; Mon, 26 Feb 2018 00:27:25 -0800 (PST) Received: from localhost.localdomain (176.122.172.82.16clouds.com. [176.122.172.82]) by smtp.gmail.com with ESMTPSA id o86sm1422706pfi.87.2018.02.26.00.27.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 26 Feb 2018 00:27:25 -0800 (PST) From: Alex Shi To: Marc Zyngier , Will Deacon , Ard Biesheuvel , Catalin Marinas , stable@vger.kernel.org, Paolo Bonzini , =?utf-8?b?UmFkaW0gS3LEjW3DocWZ?= , Christoffer Dall , Russell King , kvm@vger.kernel.org (open list:KERNEL VIRTUAL MACHINE (KVM)), linux-arm-kernel@lists.infradead.org (moderated list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), kvmarm@lists.cs.columbia.edu (open list:KERNEL VIRTUAL MACHINE (KVM) FOR ARM), linux-kernel@vger.kernel.org (open list) Subject: [PATCH 48/52] arm: KVM: Invalidate BTB on guest exit for Cortex-A12/A17 Date: Mon, 26 Feb 2018 16:20:22 +0800 Message-Id: <1519633227-29832-49-git-send-email-alex.shi@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> References: <1519633227-29832-1-git-send-email-alex.shi@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Marc Zyngier ** Not yet queued for inclusion in mainline ** In order to avoid aliasing attacks against the branch predictor, let's invalidate the BTB on guest exit. This is made complicated by the fact that we cannot take a branch before invalidating the BTB. We only apply this to A12 and A17, which are the only two ARM cores on which this useful. Signed-off-by: Marc Zyngier Signed-off-by: Will Deacon Signed-off-by: Alex Shi Conflicts: no hvc stub in hyp_hvc in arch/arm/kvm/hyp/hyp-entry.S --- arch/arm/include/asm/kvm_asm.h | 2 -- arch/arm/include/asm/kvm_mmu.h | 18 ++++++++++- arch/arm/kvm/hyp/hyp-entry.S | 71 ++++++++++++++++++++++++++++++++++++++++-- 3 files changed, 86 insertions(+), 5 deletions(-) -- 2.7.4 diff --git a/arch/arm/include/asm/kvm_asm.h b/arch/arm/include/asm/kvm_asm.h index 8ef0538..24f3ec7 100644 --- a/arch/arm/include/asm/kvm_asm.h +++ b/arch/arm/include/asm/kvm_asm.h @@ -61,8 +61,6 @@ struct kvm_vcpu; extern char __kvm_hyp_init[]; extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; - extern void __kvm_flush_vm_context(void); extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa); extern void __kvm_tlb_flush_vmid(struct kvm *kvm); diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h index d10e362..2887129 100644 --- a/arch/arm/include/asm/kvm_mmu.h +++ b/arch/arm/include/asm/kvm_mmu.h @@ -37,6 +37,7 @@ #include #include +#include #include #include @@ -225,7 +226,22 @@ static inline unsigned int kvm_get_vmid_bits(void) static inline void *kvm_get_hyp_vector(void) { - return kvm_ksym_ref(__kvm_hyp_vector); + switch(read_cpuid_part()) { +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + case ARM_CPU_PART_CORTEX_A12: + case ARM_CPU_PART_CORTEX_A17: + { + extern char __kvm_hyp_vector_bp_inv[]; + return kvm_ksym_ref(__kvm_hyp_vector_bp_inv); + } + +#endif + default: + { + extern char __kvm_hyp_vector[]; + return kvm_ksym_ref(__kvm_hyp_vector); + } + } } static inline int kvm_map_vectors(void) diff --git a/arch/arm/kvm/hyp/hyp-entry.S b/arch/arm/kvm/hyp/hyp-entry.S index 96beb53..b6b8cb1 100644 --- a/arch/arm/kvm/hyp/hyp-entry.S +++ b/arch/arm/kvm/hyp/hyp-entry.S @@ -71,6 +71,66 @@ __kvm_hyp_vector: W(b) hyp_irq W(b) hyp_fiq +#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR + .align 5 +__kvm_hyp_vector_bp_inv: + .global __kvm_hyp_vector_bp_inv + + /* + * We encode the exception entry in the bottom 3 bits of + * SP, and we have to guarantee to be 8 bytes aligned. + */ + W(add) sp, sp, #1 /* Reset 7 */ + W(add) sp, sp, #1 /* Undef 6 */ + W(add) sp, sp, #1 /* Syscall 5 */ + W(add) sp, sp, #1 /* Prefetch abort 4 */ + W(add) sp, sp, #1 /* Data abort 3 */ + W(add) sp, sp, #1 /* HVC 2 */ + W(add) sp, sp, #1 /* IRQ 1 */ + W(nop) /* FIQ 0 */ + + mcr p15, 0, r0, c7, c5, 6 /* BPIALL */ + isb + +#ifdef CONFIG_THUMB2_KERNEL + /* + * Yet another silly hack: Use VPIDR as a temp register. + * Thumb2 is really a pain, as SP cannot be used with most + * of the bitwise instructions. The vect_br macro ensures + * things gets cleaned-up. + */ + mcr p15, 4, r0, c0, c0, 0 /* VPIDR */ + mov r0, sp + and r0, r0, #7 + sub sp, sp, r0 + push {r1, r2} + mov r1, r0 + mrc p15, 4, r0, c0, c0, 0 /* VPIDR */ + mrc p15, 0, r2, c0, c0, 0 /* MIDR */ + mcr p15, 4, r2, c0, c0, 0 /* VPIDR */ +#endif + +.macro vect_br val, targ +ARM( eor sp, sp, #\val ) +ARM( tst sp, #7 ) +ARM( eorne sp, sp, #\val ) + +THUMB( cmp r1, #\val ) +THUMB( popeq {r1, r2} ) + + beq \targ +.endm + + vect_br 0, hyp_fiq + vect_br 1, hyp_irq + vect_br 2, hyp_hvc + vect_br 3, hyp_dabt + vect_br 4, hyp_pabt + vect_br 5, hyp_svc + vect_br 6, hyp_undef + vect_br 7, hyp_reset +#endif + .macro invalid_vector label, cause .align \label: mov r0, #\cause @@ -131,7 +191,14 @@ hyp_hvc: mrceq p15, 4, r0, c12, c0, 0 @ get HVBAR beq 1f - push {lr} + /* + * Pushing r2 here is just a way of keeping the stack aligned to + * 8 bytes on any path that can trigger a HYP exception. Here, + * we may well be about to jump into the guest, and the guest + * exit would otherwise be badly decoded by our fancy + * "decode-exception-without-a-branch" code... + */ + push {r2, lr} mov lr, r0 mov r0, r1 @@ -141,7 +208,7 @@ hyp_hvc: THUMB( orr lr, #1) blx lr @ Call the HYP function - pop {lr} + pop {r2, lr} 1: eret guest_trap: